Gradio

CircleGuardBench is the first-of-its-kind benchmark for evaluating the protection capabilities of large language model (LLM) guard systems.

It tests how well guard models block harmful content, resist jailbreaks, avoid false positives, and operate efficiently in real-time environments on a taxonomy close to real-world data.

Learn more about us at whitecircle.ai

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
whitecircle-policy-guard-small	Strict	Open-Source	0.726	0.931	0.930	13.954	2741.800	3920

Model

Mode

Access_Type

Integral_Score

Macro_Accuracy

Macro_Recall

Micro_Error

Micro_Avg_time_ms

Total_Count

whitecircle-policy-guard-small

Strict

Open-Source

0.726

0.931

0.930

13.954

2741.800

3920

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
whitecircle-policy-guard-small	Strict	API	0.726	0.931	0.930	0.046	908.792	3920
whitecircle-policy-guard-zero	Strict	API	0.620	0.888	0.892	0.000	127.958	3920
claude-3.5-haiku	Strict	API	0.594	0.905	0.901	0.093	2741.800	3920
gpt-4o-mini	CoT	API	0.590	0.912	0.911	0.054	3521.049	3920
gemma-3-4b-it	Strict	Open-Source	0.546	0.905	0.908	13.954	1446.367	3920
gpt-4o-mini	Strict	API	0.527	0.868	0.845	0.006	1464.863	3920
claude-3.5-haiku	CoT	API	0.523	0.916	0.909	0.233	6609.841	3920
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.496	0.852	0.833	0.000	1059.990	3920
amazon-nova-micro-v1	CoT	API	0.491	0.863	0.828	1.742	2002.835	3920
nemotron-3-content-safety	Strict	Open-Source	0.481	0.839	0.808	0.306	161.045	3920
gemini-2.0-flash-lite-001	Strict	API	0.478	0.850	0.814	0.616	1129.444	3920
claude-3.7-sonnet	Strict	API	0.475	0.851	0.819	0.132	1871.617	3920
gemma-3-12b-it	CoT	Open-Source	0.474	0.895	0.906	1.447	7366.356	3920
gemma-3-4b-it	CoT	Open-Source	0.471	0.872	0.892	3.890	3708.959	3920
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.452	0.857	0.832	0.052	3473.805	3920
llama-4-maverick	Strict	Open-Source	0.452	0.832	0.796	0.011	812.317	3920
chatgpt-4o-latest	Strict	API	0.400	0.812	0.782	0.000	1232.138	3920
openai-gpt-4.1-nano	CoT	API	0.398	0.816	0.785	0.006	1527.029	3920
qwen3-8b	Strict	Open-Source	0.398	0.805	0.777	0.000	521.935	3920
ministral-8b	Strict	Open-Source	0.396	0.808	0.759	0.058	988.065	3920
qwen3-8b	CoT	Open-Source	0.380	0.828	0.790	0.006	3781.636	3920
grok4.20	CoT	API	0.374	0.806	0.773	1.759	1844.277	3920
qwen3-4b	Strict	Open-Source	0.362	0.791	0.761	0.069	863.795	3920
gemini-2.5-flash-preview	Strict	API	0.340	0.790	0.740	0.145	1880.061	3920
gemini-2.5-flash-preview	CoT	API	0.338	0.788	0.743	0.150	1891.899	3920
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.324	0.811	0.776	6.782	3874.911	3920
o3-mini	Strict	API	0.293	0.791	0.757	7.066	3716.618	3920
grok-3-mini-beta	Strict	API	0.281	0.796	0.741	0.209	6488.239	3920
qwen3-1.7b	Strict	Open-Source	0.273	0.738	0.691	0.048	641.953	3920
llama-4-maverick	CoT	Open-Source	0.264	0.775	0.723	3.488	4679.001	3920
llama-guard-3-8b	Strict	Open-Source	0.258	0.728	0.690	0.000	526.850	3920
omni-moderation-latest	Strict	API	0.256	0.721	0.658	0.000	324.792	3920
qwen-2.5-7b-instruct	CoT	Open-Source	0.255	0.759	0.709	6.834	3112.717	3920
llama-3.1-8b-instruct	Strict	Open-Source	0.237	0.795	0.757	36.589	1116.672	3907
llama-4-scout	CoT	Open-Source	0.220	0.754	0.698	2.583	6830.184	3920
llama-guard-3-1b	Strict	Open-Source	0.216	0.692	0.667	0.000	384.449	3920
llama-guard-4-12b	Strict	Open-Source	0.208	0.693	0.643	0.000	771.426	3920
atla-selene-1-mini	Strict	Open-Source	0.206	0.723	0.663	0.000	4306.752	3920
text-moderation-latest	Strict	API	0.117	0.602	0.522	0.094	659.948	3920
llama-3.2-3b-instruct	Strict	Open-Source	0.063	0.665	0.589	63.411	932.218	3626
qwen3-0.6b	Strict	Open-Source	0.054	0.495	0.563	0.185	637.439	3920

Model

Mode

Access_Type

Integral_Score

Macro_Accuracy

Macro_Recall

Micro_Error

Micro_Avg_time_ms

Total_Count

whitecircle-policy-guard-small

Strict

API

0.726

0.931

0.930

0.046

908.792

3920

whitecircle-policy-guard-zero

Strict

API

0.620

0.888

0.892

0.000

127.958

3920

claude-3.5-haiku

Strict

API

0.594

0.905

0.901

0.093

2741.800

3920

gpt-4o-mini

CoT

API

0.590

0.912

0.911

0.054

3521.049

3920

gemma-3-4b-it

Strict

Open-Source

0.546

0.905

0.908

13.954

1446.367

3920

gpt-4o-mini

Strict

API

0.527

0.868

0.845

0.006

1464.863

3920

claude-3.5-haiku

CoT

API

0.523

0.916

0.909

0.233

6609.841

3920

mistral-small-3.1-24b-instruct

Strict

Open-Source

0.496

0.852

0.833

0.000

1059.990

3920

amazon-nova-micro-v1

CoT

API

0.491

0.863

0.828

1.742

2002.835

3920

nemotron-3-content-safety

Strict

Open-Source

0.481

0.839

0.808

0.306

161.045

3920

gemini-2.0-flash-lite-001

Strict

API

0.478

0.850

0.814

0.616

1129.444

3920

claude-3.7-sonnet

Strict

API

0.475

0.851

0.819

0.132

1871.617

3920

gemma-3-12b-it

CoT

Open-Source

0.474

0.895

0.906

1.447

7366.356

3920

gemma-3-4b-it

CoT

Open-Source

0.471

0.872

0.892

3.890

3708.959

3920

mistral-small-3.1-24b-instruct

CoT

Open-Source

0.452

0.857

0.832

0.052

3473.805

3920

llama-4-maverick

Strict

Open-Source

0.452

0.832

0.796

0.011

812.317

3920

chatgpt-4o-latest

Strict

API

0.400

0.812

0.782

0.000

1232.138

3920

openai-gpt-4.1-nano

CoT

API

0.398

0.816

0.785

0.006

1527.029

3920

qwen3-8b

Strict

Open-Source

0.398

0.805

0.777

0.000

521.935

3920

ministral-8b

Strict

Open-Source

0.396

0.808

0.759

0.058

988.065

3920

qwen3-8b

CoT

Open-Source

0.380

0.828

0.790

0.006

3781.636

3920

grok4.20

CoT

API

0.374

0.806

0.773

1.759

1844.277

3920

qwen3-4b

Strict

Open-Source

0.362

0.791

0.761

0.069

863.795

3920

gemini-2.5-flash-preview

Strict

API

0.340

0.790

0.740

0.145

1880.061

3920

gemini-2.5-flash-preview

CoT

API

0.338

0.788

0.743

0.150

1891.899

3920

deepseek-deepseek-chat-v3-0324

Strict

Open-Source

0.324

0.811

0.776

6.782

3874.911

3920

o3-mini

Strict

API

0.293

0.791

0.757

7.066

3716.618

3920

grok-3-mini-beta

Strict

API

0.281

0.796

0.741

0.209

6488.239

3920

qwen3-1.7b

Strict

Open-Source

0.273

0.738

0.691

0.048

641.953

3920

llama-4-maverick

CoT

Open-Source

0.264

0.775

0.723

3.488

4679.001

3920

llama-guard-3-8b

Strict

Open-Source

0.258

0.728

0.690

0.000

526.850

3920

omni-moderation-latest

Strict

API

0.256

0.721

0.658

0.000

324.792

3920

qwen-2.5-7b-instruct

CoT

Open-Source

0.255

0.759

0.709

6.834

3112.717

3920

llama-3.1-8b-instruct

Strict

Open-Source

0.237

0.795

0.757

36.589

1116.672

3907

llama-4-scout

CoT

Open-Source

0.220

0.754

0.698

2.583

6830.184

3920

llama-guard-3-1b

Strict

Open-Source

0.216

0.692

0.667

0.000

384.449

3920

llama-guard-4-12b

Strict

Open-Source

0.208

0.693

0.643

0.000

771.426

3920

atla-selene-1-mini

Strict

Open-Source

0.206

0.723

0.663

0.000

4306.752

3920

text-moderation-latest

Strict

API

0.117

0.602

0.522

0.094

659.948

3920

llama-3.2-3b-instruct

Strict

Open-Source

0.063

0.665

0.589

63.411

932.218

3626

qwen3-0.6b

Strict

Open-Source

0.054

0.495

0.563

0.185

637.439

3920

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
gemma-3-4b-it	CoT	Open-Source	0.615	0.891	0.958	<NA>	<NA>	120
gpt-4o-mini	CoT	API	0.595	0.883	0.883	<NA>	<NA>	120
gemma-3-12b-it	CoT	Open-Source	0.586	0.879	0.912	<NA>	<NA>	120
gemma-3-4b-it	Strict	Open-Source	0.497	0.841	0.877	<NA>	<NA>	120
whitecircle-policy-guard-small	Strict	API	0.454	0.825	0.850	<NA>	<NA>	120
claude-3.5-haiku	CoT	API	0.372	0.792	0.750	<NA>	<NA>	120
claude-3.5-haiku	Strict	API	0.366	0.783	0.750	<NA>	<NA>	120
whitecircle-policy-guard-zero	Strict	API	0.365	0.783	0.775	<NA>	<NA>	120
gpt-4o-mini	Strict	API	0.321	0.775	0.758	<NA>	<NA>	120
qwen3-0.6b	Strict	Open-Source	0.307	0.800	0.958	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.276	0.775	0.750	<NA>	<NA>	120
gemini-2.0-flash-lite-001	Strict	API	0.227	0.733	0.633	<NA>	<NA>	120
ministral-8b	Strict	Open-Source	0.207	0.717	0.633	<NA>	<NA>	120
qwen3-8b	CoT	Open-Source	0.206	0.717	0.658	<NA>	<NA>	120
amazon-nova-micro-v1	CoT	API	0.204	0.728	0.622	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.173	0.692	0.658	<NA>	<NA>	120
llama-3.1-8b-instruct	Strict	Open-Source	0.163	0.651	0.555	<NA>	<NA>	120
qwen3-8b	Strict	Open-Source	0.154	0.650	0.600	<NA>	<NA>	120
qwen3-1.7b	Strict	Open-Source	0.136	0.642	0.592	<NA>	<NA>	120
nemotron-3-content-safety	Strict	Open-Source	0.130	0.650	0.550	<NA>	<NA>	120
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.129	0.684	0.605	<NA>	<NA>	120
qwen3-4b	Strict	Open-Source	0.122	0.625	0.592	<NA>	<NA>	120
o3-mini	Strict	API	0.096	0.596	0.442	<NA>	<NA>	120
openai-gpt-4.1-nano	CoT	API	0.079	0.608	0.567	<NA>	<NA>	120
llama-4-scout	CoT	Open-Source	0.069	0.571	0.454	<NA>	<NA>	120
llama-4-maverick	Strict	Open-Source	0.063	0.592	0.508	<NA>	<NA>	120
claude-3.7-sonnet	Strict	API	0.060	0.558	0.458	<NA>	<NA>	120
qwen-2.5-7b-instruct	CoT	Open-Source	0.041	0.530	0.397	<NA>	<NA>	120
grok4.20	CoT	API	0.038	0.508	0.392	<NA>	<NA>	120
gemini-2.5-flash-preview	Strict	API	0.037	0.525	0.375	<NA>	<NA>	120
chatgpt-4o-latest	Strict	API	0.029	0.533	0.433	<NA>	<NA>	120
gemini-2.5-flash-preview	CoT	API	0.023	0.542	0.392	<NA>	<NA>	120
llama-guard-3-1b	Strict	Open-Source	0.018	0.442	0.342	<NA>	<NA>	120
llama-4-maverick	CoT	Open-Source	0.017	0.500	0.367	<NA>	<NA>	120
atla-selene-1-mini	Strict	Open-Source	0.015	0.458	0.342	<NA>	<NA>	120
grok-3-mini-beta	Strict	API	0.012	0.550	0.400	<NA>	<NA>	120
llama-guard-3-8b	Strict	Open-Source	0.005	0.375	0.242	<NA>	<NA>	120
llama-guard-4-12b	Strict	Open-Source	0.001	0.308	0.175	<NA>	<NA>	120
llama-3.2-3b-instruct	Strict	Open-Source	0.000	0.166	0.019	<NA>	<NA>	120
omni-moderation-latest	Strict	API	0.000	0.392	0.225	<NA>	<NA>	120
text-moderation-latest	Strict	API	0.000	0.167	0.000	<NA>	<NA>	120

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
gemma-3-12b-it	CoT	Open-Source	0.647	0.900	0.911	<NA>	<NA>	120
whitecircle-policy-guard-small	Strict	API	0.562	0.867	0.886	<NA>	<NA>	120
gemma-3-4b-it	CoT	Open-Source	0.505	0.853	0.874	<NA>	<NA>	120
gemma-3-4b-it	Strict	Open-Source	0.488	0.839	0.794	<NA>	<NA>	120
claude-3.5-haiku	CoT	API	0.433	0.817	0.776	<NA>	<NA>	120
gpt-4o-mini	CoT	API	0.405	0.808	0.813	<NA>	<NA>	120
claude-3.5-haiku	Strict	API	0.381	0.792	0.722	<NA>	<NA>	120
whitecircle-policy-guard-zero	Strict	API	0.345	0.775	0.748	<NA>	<NA>	120
amazon-nova-micro-v1	CoT	API	0.303	0.774	0.689	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.263	0.750	0.666	<NA>	<NA>	120
claude-3.7-sonnet	Strict	API	0.255	0.733	0.661	<NA>	<NA>	120
qwen3-0.6b	Strict	Open-Source	0.243	0.742	0.892	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.237	0.717	0.639	<NA>	<NA>	120
llama-4-maverick	Strict	Open-Source	0.225	0.725	0.667	<NA>	<NA>	120
qwen3-8b	CoT	Open-Source	0.199	0.717	0.653	<NA>	<NA>	120
gpt-4o-mini	Strict	API	0.179	0.683	0.620	<NA>	<NA>	120
gemini-2.0-flash-lite-001	Strict	API	0.173	0.692	0.614	<NA>	<NA>	120
qwen3-8b	Strict	Open-Source	0.140	0.667	0.655	<NA>	<NA>	120
grok4.20	CoT	API	0.139	0.642	0.555	<NA>	<NA>	120
llama-3.1-8b-instruct	Strict	Open-Source	0.135	0.624	0.533	<NA>	<NA>	120
openai-gpt-4.1-nano	CoT	API	0.126	0.658	0.583	<NA>	<NA>	120
nemotron-3-content-safety	Strict	Open-Source	0.119	0.650	0.564	<NA>	<NA>	120
ministral-8b	Strict	Open-Source	0.118	0.633	0.541	<NA>	<NA>	120
llama-4-maverick	CoT	Open-Source	0.093	0.642	0.555	<NA>	<NA>	120
qwen3-4b	Strict	Open-Source	0.090	0.608	0.559	<NA>	<NA>	120
chatgpt-4o-latest	Strict	API	0.066	0.583	0.497	<NA>	<NA>	120
gemini-2.5-flash-preview	CoT	API	0.066	0.583	0.491	<NA>	<NA>	120
qwen3-1.7b	Strict	Open-Source	0.061	0.583	0.536	<NA>	<NA>	120
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.061	0.597	0.512	<NA>	<NA>	120
llama-guard-3-1b	Strict	Open-Source	0.050	0.525	0.464	<NA>	<NA>	120
grok-3-mini-beta	Strict	API	0.049	0.589	0.462	<NA>	<NA>	120
llama-4-scout	CoT	Open-Source	0.047	0.581	0.450	<NA>	<NA>	120
qwen-2.5-7b-instruct	CoT	Open-Source	0.043	0.545	0.459	<NA>	<NA>	120
gemini-2.5-flash-preview	Strict	API	0.039	0.583	0.497	<NA>	<NA>	120
o3-mini	Strict	API	0.027	0.530	0.484	<NA>	<NA>	120
llama-guard-3-8b	Strict	Open-Source	0.024	0.508	0.453	<NA>	<NA>	120
atla-selene-1-mini	Strict	Open-Source	0.018	0.508	0.422	<NA>	<NA>	120
llama-guard-4-12b	Strict	Open-Source	0.017	0.492	0.391	<NA>	<NA>	120
omni-moderation-latest	Strict	API	0.007	0.475	0.331	<NA>	<NA>	120
llama-3.2-3b-instruct	Strict	Open-Source	0.000	0.280	0.157	<NA>	<NA>	120
text-moderation-latest	Strict	API	0.000	0.158	0.000	<NA>	<NA>	120

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
claude-3.5-haiku	CoT	API	0.543	0.865	0.900	<NA>	<NA>	120
gemma-3-12b-it	CoT	Open-Source	0.499	0.857	0.934	<NA>	<NA>	120
amazon-nova-micro-v1	CoT	API	0.444	0.824	0.844	<NA>	<NA>	120
claude-3.5-haiku	Strict	API	0.440	0.821	0.840	<NA>	<NA>	120
gemma-3-4b-it	CoT	Open-Source	0.440	0.839	0.930	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.421	0.816	0.846	<NA>	<NA>	120
gpt-4o-mini	CoT	API	0.381	0.792	0.821	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.376	0.792	0.861	<NA>	<NA>	120
whitecircle-policy-guard-small	Strict	API	0.365	0.792	0.786	<NA>	<NA>	120
gemma-3-4b-it	Strict	Open-Source	0.322	0.761	0.781	<NA>	<NA>	120
claude-3.7-sonnet	Strict	API	0.270	0.749	0.758	<NA>	<NA>	120
qwen3-0.6b	Strict	Open-Source	0.245	0.732	0.829	<NA>	<NA>	120
nemotron-3-content-safety	Strict	Open-Source	0.244	0.715	0.692	<NA>	<NA>	120
gpt-4o-mini	Strict	API	0.244	0.717	0.713	<NA>	<NA>	120
qwen3-8b	CoT	Open-Source	0.213	0.708	0.651	<NA>	<NA>	120
grok-3-mini-beta	Strict	API	0.211	0.707	0.650	<NA>	<NA>	120
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.202	0.709	0.703	<NA>	<NA>	120
chatgpt-4o-latest	Strict	API	0.200	0.675	0.671	<NA>	<NA>	120
whitecircle-policy-guard-zero	Strict	API	0.199	0.683	0.663	<NA>	<NA>	120
openai-gpt-4.1-nano	CoT	API	0.180	0.708	0.659	<NA>	<NA>	120
ministral-8b	Strict	Open-Source	0.155	0.650	0.607	<NA>	<NA>	120
llama-3.1-8b-instruct	Strict	Open-Source	0.144	0.623	0.571	<NA>	<NA>	120
qwen-2.5-7b-instruct	CoT	Open-Source	0.124	0.615	0.566	<NA>	<NA>	120
qwen3-8b	Strict	Open-Source	0.118	0.625	0.596	<NA>	<NA>	120
llama-4-maverick	Strict	Open-Source	0.108	0.625	0.598	<NA>	<NA>	120
gemini-2.5-flash-preview	Strict	API	0.100	0.623	0.570	<NA>	<NA>	120
gemini-2.5-flash-preview	CoT	API	0.089	0.604	0.537	<NA>	<NA>	120
gemini-2.0-flash-lite-001	Strict	API	0.089	0.584	0.528	<NA>	<NA>	120
qwen3-4b	Strict	Open-Source	0.078	0.572	0.523	<NA>	<NA>	120
grok4.20	CoT	API	0.063	0.526	0.471	<NA>	<NA>	120
qwen3-1.7b	Strict	Open-Source	0.048	0.492	0.422	<NA>	<NA>	120
llama-4-scout	CoT	Open-Source	0.038	0.524	0.426	<NA>	<NA>	120
llama-4-maverick	CoT	Open-Source	0.035	0.522	0.423	<NA>	<NA>	120
llama-guard-4-12b	Strict	Open-Source	0.026	0.433	0.364	<NA>	<NA>	120
o3-mini	Strict	API	0.025	0.444	0.380	<NA>	<NA>	120
llama-guard-3-1b	Strict	Open-Source	0.015	0.400	0.367	<NA>	<NA>	120
atla-selene-1-mini	Strict	Open-Source	0.010	0.392	0.277	<NA>	<NA>	120
llama-guard-3-8b	Strict	Open-Source	0.009	0.358	0.266	<NA>	<NA>	120
omni-moderation-latest	Strict	API	0.001	0.275	0.145	<NA>	<NA>	120
text-moderation-latest	Strict	API	0.000	0.183	0.025	<NA>	<NA>	120
llama-3.2-3b-instruct	Strict	Open-Source	0.000	0.228	0.186	<NA>	<NA>	120

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
gemma-3-12b-it	CoT	Open-Source	0.700	0.921	0.929	<NA>	<NA>	120
gemma-3-4b-it	Strict	Open-Source	0.688	0.913	0.923	<NA>	<NA>	120
gemma-3-4b-it	CoT	Open-Source	0.684	0.914	0.966	<NA>	<NA>	120
whitecircle-policy-guard-small	Strict	API	0.534	0.858	0.852	<NA>	<NA>	120
claude-3.5-haiku	Strict	API	0.517	0.850	0.861	<NA>	<NA>	120
claude-3.5-haiku	CoT	API	0.485	0.842	0.852	<NA>	<NA>	120
whitecircle-policy-guard-zero	Strict	API	0.475	0.833	0.875	<NA>	<NA>	120
gpt-4o-mini	CoT	API	0.340	0.775	0.748	<NA>	<NA>	120
claude-3.7-sonnet	Strict	API	0.282	0.748	0.705	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.245	0.750	0.707	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.231	0.725	0.696	<NA>	<NA>	120
nemotron-3-content-safety	Strict	Open-Source	0.228	0.717	0.659	<NA>	<NA>	120
amazon-nova-micro-v1	CoT	API	0.216	0.744	0.701	<NA>	<NA>	120
qwen3-0.6b	Strict	Open-Source	0.207	0.741	0.870	<NA>	<NA>	120
gpt-4o-mini	Strict	API	0.180	0.683	0.626	<NA>	<NA>	120
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.177	0.691	0.628	<NA>	<NA>	120
gemini-2.0-flash-lite-001	Strict	API	0.170	0.716	0.673	<NA>	<NA>	120
qwen3-8b	CoT	Open-Source	0.157	0.683	0.583	<NA>	<NA>	120
llama-4-maverick	Strict	Open-Source	0.148	0.650	0.592	<NA>	<NA>	120
ministral-8b	Strict	Open-Source	0.138	0.650	0.549	<NA>	<NA>	120
qwen3-8b	Strict	Open-Source	0.137	0.658	0.601	<NA>	<NA>	120
grok4.20	CoT	API	0.129	0.649	0.620	<NA>	<NA>	120
chatgpt-4o-latest	Strict	API	0.124	0.650	0.621	<NA>	<NA>	120
o3-mini	Strict	API	0.115	0.628	0.608	<NA>	<NA>	120
openai-gpt-4.1-nano	CoT	API	0.106	0.625	0.564	<NA>	<NA>	120
llama-guard-3-1b	Strict	Open-Source	0.102	0.600	0.596	<NA>	<NA>	120
qwen3-4b	Strict	Open-Source	0.098	0.592	0.514	<NA>	<NA>	120
qwen3-1.7b	Strict	Open-Source	0.092	0.592	0.491	<NA>	<NA>	120
gemini-2.5-flash-preview	Strict	API	0.067	0.632	0.560	<NA>	<NA>	120
qwen-2.5-7b-instruct	CoT	Open-Source	0.059	0.543	0.450	<NA>	<NA>	120
llama-3.1-8b-instruct	Strict	Open-Source	0.056	0.533	0.505	<NA>	<NA>	120
gemini-2.5-flash-preview	CoT	API	0.053	0.558	0.443	<NA>	<NA>	120
llama-4-maverick	CoT	Open-Source	0.046	0.539	0.426	<NA>	<NA>	120
llama-guard-3-8b	Strict	Open-Source	0.038	0.508	0.451	<NA>	<NA>	120
llama-4-scout	CoT	Open-Source	0.035	0.507	0.407	<NA>	<NA>	120
grok-3-mini-beta	Strict	API	0.033	0.558	0.443	<NA>	<NA>	120
llama-guard-4-12b	Strict	Open-Source	0.023	0.458	0.389	<NA>	<NA>	120
atla-selene-1-mini	Strict	Open-Source	0.021	0.467	0.337	<NA>	<NA>	120
omni-moderation-latest	Strict	API	0.012	0.383	0.287	<NA>	<NA>	120
llama-3.2-3b-instruct	Strict	Open-Source	0.000	0.212	0.083	<NA>	<NA>	120
text-moderation-latest	Strict	API	0.000	0.201	0.043	<NA>	<NA>	120

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
gemma-3-12b-it	CoT	Open-Source	0.711	0.922	0.948	<NA>	<NA>	116
whitecircle-policy-guard-small	Strict	API	0.620	0.888	0.922	<NA>	<NA>	116
gemma-3-4b-it	CoT	Open-Source	0.576	0.879	0.957	<NA>	<NA>	116
gemma-3-4b-it	Strict	Open-Source	0.499	0.846	0.915	<NA>	<NA>	116
claude-3.5-haiku	Strict	API	0.379	0.793	0.810	<NA>	<NA>	116
whitecircle-policy-guard-zero	Strict	API	0.377	0.784	0.836	<NA>	<NA>	116
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.363	0.784	0.828	<NA>	<NA>	116
gpt-4o-mini	CoT	API	0.342	0.784	0.802	<NA>	<NA>	116
claude-3.5-haiku	CoT	API	0.337	0.776	0.793	<NA>	<NA>	116
qwen3-0.6b	Strict	Open-Source	0.220	0.767	0.948	<NA>	<NA>	116
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.213	0.741	0.668	<NA>	<NA>	116
amazon-nova-micro-v1	CoT	API	0.212	0.735	0.669	<NA>	<NA>	116
gpt-4o-mini	Strict	API	0.171	0.672	0.627	<NA>	<NA>	116
qwen3-8b	CoT	Open-Source	0.165	0.707	0.608	<NA>	<NA>	116
llama-3.1-8b-instruct	Strict	Open-Source	0.163	0.658	0.570	<NA>	<NA>	116
gemini-2.0-flash-lite-001	Strict	API	0.153	0.672	0.636	<NA>	<NA>	116
nemotron-3-content-safety	Strict	Open-Source	0.148	0.664	0.610	<NA>	<NA>	116
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.136	0.690	0.699	<NA>	<NA>	116
ministral-8b	Strict	Open-Source	0.115	0.638	0.556	<NA>	<NA>	116
qwen3-8b	Strict	Open-Source	0.107	0.664	0.636	<NA>	<NA>	116
llama-4-maverick	Strict	Open-Source	0.105	0.638	0.601	<NA>	<NA>	116
claude-3.7-sonnet	Strict	API	0.104	0.638	0.584	<NA>	<NA>	116
qwen3-4b	Strict	Open-Source	0.083	0.603	0.584	<NA>	<NA>	116
grok4.20	CoT	API	0.072	0.578	0.524	<NA>	<NA>	116
qwen3-1.7b	Strict	Open-Source	0.063	0.593	0.565	<NA>	<NA>	116
qwen-2.5-7b-instruct	CoT	Open-Source	0.062	0.577	0.486	<NA>	<NA>	116
o3-mini	Strict	API	0.060	0.580	0.588	<NA>	<NA>	116
chatgpt-4o-latest	Strict	API	0.058	0.586	0.595	<NA>	<NA>	116
openai-gpt-4.1-nano	CoT	API	0.046	0.603	0.584	<NA>	<NA>	116
omni-moderation-latest	Strict	API	0.040	0.517	0.364	<NA>	<NA>	116
gemini-2.5-flash-preview	CoT	API	0.039	0.586	0.586	<NA>	<NA>	116
grok-3-mini-beta	Strict	API	0.033	0.609	0.448	<NA>	<NA>	116
llama-guard-3-1b	Strict	Open-Source	0.027	0.483	0.401	<NA>	<NA>	116
llama-4-scout	CoT	Open-Source	0.023	0.514	0.353	<NA>	<NA>	116
gemini-2.5-flash-preview	Strict	API	0.014	0.578	0.578	<NA>	<NA>	116
atla-selene-1-mini	Strict	Open-Source	0.013	0.509	0.401	<NA>	<NA>	116
llama-guard-3-8b	Strict	Open-Source	0.010	0.448	0.412	<NA>	<NA>	116
llama-4-maverick	CoT	Open-Source	0.009	0.509	0.348	<NA>	<NA>	116
llama-guard-4-12b	Strict	Open-Source	0.002	0.388	0.343	<NA>	<NA>	116
llama-3.2-3b-instruct	Strict	Open-Source	0.000	0.225	0.013	<NA>	<NA>	116
text-moderation-latest	Strict	API	0.000	0.292	0.078	<NA>	<NA>	116

CircleGuardBench Leaderboard

Submit Your Model

✉️✨ Ready? Upload your results below!

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
gpt-4o-mini	CoT	API	0.754	0.933	0.923	<NA>	<NA>	120
gemma-3-12b-it	CoT	Open-Source	0.708	0.921	0.920	<NA>	<NA>	120
whitecircle-policy-guard-small	Strict	API	0.668	0.908	0.889	<NA>	<NA>	120
gemma-3-4b-it	CoT	Open-Source	0.640	0.903	0.938	<NA>	<NA>	120
gemma-3-4b-it	Strict	Open-Source	0.529	0.856	0.876	<NA>	<NA>	120
claude-3.5-haiku	Strict	API	0.529	0.858	0.848	<NA>	<NA>	120
claude-3.5-haiku	CoT	API	0.500	0.850	0.749	<NA>	<NA>	120
gpt-4o-mini	Strict	API	0.447	0.825	0.751	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.326	0.792	0.726	<NA>	<NA>	120
whitecircle-policy-guard-zero	Strict	API	0.306	0.767	0.701	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.259	0.725	0.613	<NA>	<NA>	120
gemini-2.0-flash-lite-001	Strict	API	0.258	0.742	0.640	<NA>	<NA>	120
claude-3.7-sonnet	Strict	API	0.235	0.725	0.643	<NA>	<NA>	120
o3-mini	Strict	API	0.225	0.723	0.610	<NA>	<NA>	120
qwen3-0.6b	Strict	Open-Source	0.216	0.767	0.950	<NA>	<NA>	120
ministral-8b	Strict	Open-Source	0.204	0.708	0.607	<NA>	<NA>	120
amazon-nova-micro-v1	CoT	API	0.203	0.714	0.602	<NA>	<NA>	120
llama-4-maverick	Strict	Open-Source	0.201	0.708	0.599	<NA>	<NA>	120
grok4.20	CoT	API	0.186	0.678	0.541	<NA>	<NA>	120
nemotron-3-content-safety	Strict	Open-Source	0.183	0.692	0.555	<NA>	<NA>	120
qwen3-8b	Strict	Open-Source	0.147	0.658	0.557	<NA>	<NA>	120
chatgpt-4o-latest	Strict	API	0.121	0.642	0.505	<NA>	<NA>	120
qwen3-8b	CoT	Open-Source	0.119	0.650	0.530	<NA>	<NA>	120
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.117	0.659	0.527	<NA>	<NA>	120
llama-guard-3-1b	Strict	Open-Source	0.110	0.600	0.515	<NA>	<NA>	120
qwen3-4b	Strict	Open-Source	0.107	0.633	0.532	<NA>	<NA>	120
openai-gpt-4.1-nano	CoT	API	0.095	0.633	0.500	<NA>	<NA>	120
llama-3.1-8b-instruct	Strict	Open-Source	0.094	0.595	0.486	<NA>	<NA>	120
qwen-2.5-7b-instruct	CoT	Open-Source	0.075	0.581	0.471	<NA>	<NA>	120
gemini-2.5-flash-preview	CoT	API	0.074	0.592	0.400	<NA>	<NA>	120
grok-3-mini-beta	Strict	API	0.066	0.608	0.444	<NA>	<NA>	120
llama-4-maverick	CoT	Open-Source	0.057	0.582	0.391	<NA>	<NA>	120
gemini-2.5-flash-preview	Strict	API	0.055	0.558	0.367	<NA>	<NA>	120
qwen3-1.7b	Strict	Open-Source	0.052	0.575	0.411	<NA>	<NA>	120
llama-4-scout	CoT	Open-Source	0.051	0.558	0.394	<NA>	<NA>	120
llama-guard-3-8b	Strict	Open-Source	0.031	0.500	0.352	<NA>	<NA>	120
atla-selene-1-mini	Strict	Open-Source	0.009	0.475	0.311	<NA>	<NA>	120
llama-guard-4-12b	Strict	Open-Source	0.007	0.442	0.305	<NA>	<NA>	120
llama-3.2-3b-instruct	Strict	Open-Source	0.000	0.272	0.012	<NA>	<NA>	90
omni-moderation-latest	Strict	API	0.000	0.517	0.325	<NA>	<NA>	120
text-moderation-latest	Strict	API	0.000	0.192	0.000	<NA>	<NA>	120

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
gemma-3-12b-it	CoT	Open-Source	0.745	0.932	0.957	<NA>	<NA>	120
whitecircle-policy-guard-small	Strict	API	0.722	0.925	0.906	<NA>	<NA>	120
gemma-3-4b-it	CoT	Open-Source	0.629	0.896	0.956	<NA>	<NA>	120
gemma-3-4b-it	Strict	Open-Source	0.595	0.884	0.901	<NA>	<NA>	120
gpt-4o-mini	CoT	API	0.532	0.867	0.848	<NA>	<NA>	120
claude-3.5-haiku	CoT	API	0.472	0.833	0.823	<NA>	<NA>	120
whitecircle-policy-guard-zero	Strict	API	0.453	0.825	0.858	<NA>	<NA>	120
claude-3.5-haiku	Strict	API	0.395	0.808	0.789	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.265	0.767	0.701	<NA>	<NA>	120
amazon-nova-micro-v1	CoT	API	0.252	0.757	0.700	<NA>	<NA>	120
gpt-4o-mini	Strict	API	0.248	0.733	0.679	<NA>	<NA>	120
claude-3.7-sonnet	Strict	API	0.226	0.733	0.651	<NA>	<NA>	120
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.223	0.725	0.651	<NA>	<NA>	120
nemotron-3-content-safety	Strict	Open-Source	0.211	0.708	0.662	<NA>	<NA>	120
llama-4-maverick	Strict	Open-Source	0.173	0.700	0.590	<NA>	<NA>	120
qwen3-4b	Strict	Open-Source	0.162	0.692	0.664	<NA>	<NA>	120
qwen3-0.6b	Strict	Open-Source	0.156	0.742	0.870	<NA>	<NA>	120
gemini-2.0-flash-lite-001	Strict	API	0.138	0.717	0.635	<NA>	<NA>	120
qwen3-8b	CoT	Open-Source	0.135	0.700	0.700	<NA>	<NA>	120
qwen3-8b	Strict	Open-Source	0.106	0.675	0.648	<NA>	<NA>	120
openai-gpt-4.1-nano	CoT	API	0.102	0.683	0.645	<NA>	<NA>	120
llama-guard-3-1b	Strict	Open-Source	0.095	0.600	0.598	<NA>	<NA>	120
ministral-8b	Strict	Open-Source	0.091	0.642	0.505	<NA>	<NA>	120
gemini-2.5-flash-preview	CoT	API	0.090	0.625	0.461	<NA>	<NA>	120
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.087	0.651	0.543	<NA>	<NA>	120
grok4.20	CoT	API	0.082	0.600	0.518	<NA>	<NA>	120
llama-3.1-8b-instruct	Strict	Open-Source	0.080	0.622	0.583	<NA>	<NA>	120
chatgpt-4o-latest	Strict	API	0.070	0.583	0.474	<NA>	<NA>	120
llama-4-maverick	CoT	Open-Source	0.057	0.610	0.501	<NA>	<NA>	120
gemini-2.5-flash-preview	Strict	API	0.054	0.597	0.460	<NA>	<NA>	120
grok-3-mini-beta	Strict	API	0.045	0.617	0.535	<NA>	<NA>	120
llama-guard-4-12b	Strict	Open-Source	0.044	0.517	0.407	<NA>	<NA>	120
qwen3-1.7b	Strict	Open-Source	0.036	0.567	0.430	<NA>	<NA>	120
qwen-2.5-7b-instruct	CoT	Open-Source	0.025	0.538	0.428	<NA>	<NA>	120
llama-guard-3-8b	Strict	Open-Source	0.017	0.483	0.429	<NA>	<NA>	120
o3-mini	Strict	API	0.010	0.470	0.333	<NA>	<NA>	120
llama-4-scout	CoT	Open-Source	0.008	0.527	0.395	<NA>	<NA>	120
omni-moderation-latest	Strict	API	0.006	0.458	0.294	<NA>	<NA>	120
atla-selene-1-mini	Strict	Open-Source	0.006	0.492	0.355	<NA>	<NA>	120
llama-3.2-3b-instruct	Strict	Open-Source	0.000	0.336	0.188	<NA>	<NA>	90
text-moderation-latest	Strict	API	0.000	0.192	0.000	<NA>	<NA>	120

Model	Mode	Access_Type	Integral_Score	Macro_Accuracy	Macro_Recall	Micro_Error	Micro_Avg_time_ms	Total_Count
o3-mini	Strict	API	1.000	1.000	1.000	<NA>	<NA>	1960
text-moderation-latest	Strict	API	1.000	1.000	1.000	<NA>	<NA>	1960
chatgpt-4o-latest	Strict	API	0.988	0.997	0.997	<NA>	<NA>	1960
gemini-2.5-flash-preview	Strict	API	0.988	0.997	0.997	<NA>	<NA>	1960
gemini-2.0-flash-lite-001	Strict	API	0.988	0.997	0.997	<NA>	<NA>	1960
gpt-4o-mini	Strict	API	0.988	0.997	0.997	<NA>	<NA>	1960
llama-4-maverick	CoT	Open-Source	0.988	0.997	0.997	<NA>	<NA>	1960
gemini-2.5-flash-preview	CoT	API	0.984	0.996	0.996	<NA>	<NA>	1960
atla-selene-1-mini	Strict	Open-Source	0.984	0.996	0.996	<NA>	<NA>	1960
whitecircle-policy-guard-zero	Strict	API	0.980	0.995	0.995	<NA>	<NA>	1960
llama-4-scout	CoT	Open-Source	0.979	0.995	0.995	<NA>	<NA>	1960
claude-3.5-haiku	Strict	API	0.976	0.994	0.994	<NA>	<NA>	1960
claude-3.5-haiku	CoT	API	0.972	0.993	0.993	<NA>	<NA>	1960
grok-3-mini-beta	Strict	API	0.972	0.993	0.993	<NA>	<NA>	1960
openai-gpt-4.1-nano	CoT	API	0.972	0.993	0.993	<NA>	<NA>	1960
claude-3.7-sonnet	Strict	API	0.968	0.992	0.992	<NA>	<NA>	1960
qwen3-8b	Strict	Open-Source	0.960	0.990	0.990	<NA>	<NA>	1960
whitecircle-policy-guard-small	Strict	API	0.956	0.989	0.989	<NA>	<NA>	1960
llama-guard-3-8b	Strict	Open-Source	0.956	0.989	0.989	<NA>	<NA>	1960
nemotron-3-content-safety	Strict	Open-Source	0.956	0.989	0.989	<NA>	<NA>	1960
llama-4-maverick	Strict	Open-Source	0.956	0.989	0.989	<NA>	<NA>	1960
ministral-8b	Strict	Open-Source	0.956	0.989	0.989	<NA>	<NA>	1960
qwen3-4b	Strict	Open-Source	0.955	0.989	0.989	<NA>	<NA>	1960
qwen-2.5-7b-instruct	CoT	Open-Source	0.955	0.989	0.989	<NA>	<NA>	1960
grok4.20	CoT	API	0.944	0.986	0.986	<NA>	<NA>	1960
gpt-4o-mini	CoT	API	0.928	0.982	0.982	<NA>	<NA>	1960
llama-3.1-8b-instruct	Strict	Open-Source	0.925	0.981	0.981	<NA>	<NA>	1960
mistral-small-3.1-24b-instruct	Strict	Open-Source	0.921	0.980	0.980	<NA>	<NA>	1960
deepseek-deepseek-chat-v3-0324	Strict	Open-Source	0.916	0.978	0.978	<NA>	<NA>	1960
qwen3-8b	CoT	Open-Source	0.897	0.973	0.973	<NA>	<NA>	1960
omni-moderation-latest	Strict	API	0.890	0.971	0.971	<NA>	<NA>	1960
amazon-nova-micro-v1	CoT	API	0.862	0.964	0.964	<NA>	<NA>	1960
gemma-3-4b-it	Strict	Open-Source	0.851	0.961	0.961	<NA>	<NA>	1960
mistral-small-3.1-24b-instruct	CoT	Open-Source	0.833	0.956	0.956	<NA>	<NA>	1960
llama-3.2-3b-instruct	Strict	Open-Source	0.824	0.954	0.954	<NA>	<NA>	1960
llama-guard-4-12b	Strict	Open-Source	0.797	0.945	0.945	<NA>	<NA>	1960
qwen3-1.7b	Strict	Open-Source	0.780	0.941	0.941	<NA>	<NA>	1960
gemma-3-12b-it	CoT	Open-Source	0.577	0.877	0.877	<NA>	<NA>	1960
gemma-3-4b-it	CoT	Open-Source	0.500	0.849	0.849	<NA>	<NA>	1960
llama-guard-3-1b	Strict	Open-Source	0.471	0.829	0.829	<NA>	<NA>	1960
qwen3-0.6b	Strict	Open-Source	0.002	0.227	0.227	<NA>	<NA>	1960