Benchmark Almanac

BullshitBench V2

Claude Opus 4.8 95% leads since May 2026
Not in model cards

Models answer 100 nonsense questions across software, finance, legal, medical and physics, built with 13 techniques. A three-judge panel scores clear pushback.

Released March 1st, 2026 with a new question set, so scores do not compare with V1. 236 model variants by October 13th, 2026.

Made by
Peter Gostev
Released
March 2026
Kind
Hallucination rates
Level
Everyday
Scored
Graded by a model
Answers
Open-ended
Tasks
All public
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last model added
November 2026
Scores refreshed
November 12th, 2026
137
Entries on its own leaderboard, 129 models we track

2 models share the top score of 95%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

137 entries. Scroll sideways for the rest.

  1. 95.0 Claude Opus 4.8
  2. 95.0 Qwen 3.8 Max
  3. 91.0 Claude Sonnet 4.6
  4. 90.0 Claude Opus 4.5
  5. 87.0 Claude Opus 4.6
  6. 86.5 Claude Fable 5.1
  7. 83.6 Claude Fable 5
  8. 83.0 Claude Opus 4.7
  9. 80.8 Claude Sonnet 5
  10. 80.2 Claude Opus 5
  11. 80.0 Mistral Large 4
  12. 79.0 Claude Sonnet 4.5
  13. 79.0 Qwen 3.8 27B
  14. 78.0 Qwen3.5 397B-A17B
  15. 77.0 Claude Haiku 4.5
  16. 75.0 Grok 4.7
  17. 74.0 Kimi K3
  18. 72.0 Qwen 3.6 Plus
  19. 72.0 Qwen3.7-Max
  20. 72.0 GLM-5.3
  21. 72.0 Gemini 4 Carbon
  22. 69.2 Claude Opus 5.5
  23. 69.0 GPT-6 Astra
  24. 67.3 Claude Fable 5.2
  25. 67.0 Grok 4.20 Multi-agent
  26. 66.0 Gemini 3.5 Flash-Lite
  27. 66.0 Grok 4.6
  28. 66.0 Gemini 4 Argon
  29. 65.0 stealth/ox-alpha
  30. 65.0 Kimi K2.6
  31. 65.0 GPT-6.1 Sol
  32. 64.0 MiniMax-M3
  33. 63.0 Qwen3-Max
  34. 62.6 Muse Ember
  35. 62.0 MiMo-V2.5-Pro
  36. 60.0 Claude Sonnet 5.5
  37. 60.0 Kimi K3.1 Pro
  38. 59.0 openai/gpt-6-luna-pro
  39. 59.0 GPT-6 Luna
  40. 59.0 GPT-6 Sol
  41. 57.6 Claude Haiku 5.5
  42. 57.6 Muse Wonder
  43. 57.0 openai/gpt-6-sol-pro
  44. 57.0 DeepSeek V4.1 Flash
  45. 57.0 Tencent Hy4
  46. 56.0 Grok 4.20
  47. 55.0 Grok 4.5
  48. 54.0 Nemotron 3 Super
  49. 54.0 GPT-5.6 Terra
  50. 54.0 GPT-6.1 Terra
  51. 52.0 Kimi K2.5
  52. 50.0 Claude 3.5 Haiku
  53. 50.0 Grok 4.3
  54. 50.0 Muse Spark 1.2
  55. 49.0 Claude 3.7 Sonnet
  56. 49.0 Nemotron 3 Ultra
  57. 48.0 Gemini 3 Pro
  58. 48.0 GPT-5.4
  59. 48.0 GPT-5.6 Sol
  60. 47.0 GPT-5.5
  61. 46.0 Claude 3.5 Sonnet
  62. 45.0 GPT-5.1 Instant
  63. 45.0 GPT-5.2 Codex
  64. 43.0 openrouter/hunter-alpha
  65. 43.0 Claude Opus 4.1
  66. 41.0 GPT-5.6 Luna
  67. 40.0 GPT-5.3 Instant
  68. 39.0 GPT-5-Codex
  69. 39.0 Gemini 3.6 Flash
  70. 39.0 DeepSeek V4 Flash 0731
  71. 38.0 GPT-5.2
  72. 37.0 openrouter/healer-alpha
  73. 37.0 Gemini 3.1 Pro
  74. 36.0 GPT-5.5 Pro
  75. 35.0 MiMo-V2.5
  76. 35.0 DeepSeek V4 Pro 0813
  77. 35.0 Gemini 3.7 Flash
  78. 34.0 openai/gpt-5.5-chat
  79. 34.0 Claude Opus 4
  80. 32.0 GPT-5.1-Codex
  81. 32.0 GPT-5.4 Mini
  82. 31.0 GLM-5-Turbo
  83. 31.0 GLM-5.2
  84. 30.0 Claude Sonnet 4
  85. 29.0 Llama 4 Maverick
  86. 28.0 Nemotron 3 Nano 30B-A3B
  87. 28.0 GLM-5
  88. 27.0 GPT-5.2 Instant
  89. 26.0 o3
  90. 25.0 GPT-5.1
  91. 25.0 Gemma 4 31B IT
  92. 24.0 GPT-5.3 Codex
  93. 22.0 Step 3.5 Flash
  94. 22.0 GLM-5.1
  95. 22.0 Laguna S 2.1
  96. 21.0 GPT-5
  97. 21.0 Gemma 4 26B A4B
  98. 20.0 Llama 4 Scout
  99. 20.0 Gemini 2.5 Pro
  100. 20.0 Qwen3-Coder-480B-A35B
  101. 20.0 Gemini 3.5 Flash
  102. 19.0 Gemini 2.5 Flash
  103. 19.0 Grok 4.1 Fast
  104. 18.0 GPT-5 Chat
  105. 18.0 DeepSeek-V4-Flash
  106. 17.0 Trinity Large Thinking
  107. 16.0 MiMo-V2-Flash
  108. 16.0 Tencent Hy3 preview
  109. 15.0 Gemini 2.0 Flash
  110. 14.0 Llama 3.1-8B
  111. 14.0 GPT-4.1
  112. 14.0 GPT-5.4 Nano
  113. 14.0 DeepSeek-V4-Pro
  114. 13.0 DeepSeek-V3.2
  115. 12.0 GPT-4o
  116. 12.0 gpt-oss-120b
  117. 11.0 baidu/ernie-4.5-vl-424b-a47b
  118. 11.0 Doubao-Seed-1.6
  119. 11.0 Gemini 3.1 Flash-Lite
  120. 10.0 Claude 3 Haiku
  121. 10.0 Kimi K2
  122. 10.0 Gemini 3 Flash
  123. 9.00 MiniMax-M2.5
  124. 8.00 o4-mini
  125. 8.00 DeepSeek-R1
  126. 8.00 GLM-4.5
  127. 8.00 MiniMax-M2.7
  128. 7.00 INTELLECT-3
  129. 6.00 Qwen3-235B-A22B
  130. 6.00 Mistral Small 4
  131. 5.00 Jamba 1.7 Large
  132. 5.00 nvidia-nemotron-nano-9b-v2
  133. 4.00 deepseek/deepseek-chat
  134. 4.00 ERNIE-4.5-300B-A47B
  135. 3.00 Gemma 3 27B
  136. 2.00 GPT-4o mini
  137. 2.00 Mistral Large 3
  1. 1 Claude Opus 4.8 none 95%
  2. 2 Qwen 3.8 Max minimal 95%
  3. 3 Claude Sonnet 4.6 high 91%
  4. 4 Claude Opus 4.5 high 90%
  5. 5 Claude Opus 4.6 high 87%
  6. 6 Claude Fable 5.1 low 86.5%
  7. 7 Claude Fable 5 xhigh 83.6%
  8. 8 Claude Opus 4.7 none 83%
  9. 9 Claude Sonnet 5 low 80.8%
  10. 10 Claude Opus 5 low 80.2%
  11. 11 Mistral Large 4 none 80%
  12. 12 Claude Sonnet 4.5 high 79%
  13. 13 Qwen 3.8 27B none 79%
  14. 14 Qwen3.5 397B-A17B high 78%
  15. 15 Claude Haiku 4.5 high 77%
  16. 16 Grok 4.7 low 75%
  17. 17 Kimi K3 xhigh 74%
  18. 18 Qwen 3.6 Plus none 72%
  19. 19 Qwen3.7-Max none 72%
  20. 20 GLM-5.3 minimal 72%
  21. 21 Gemini 4 Carbon high 72%
  22. 22 Claude Opus 5.5 low 69.2%
  23. 23 GPT-6 Astra max 69%
  24. 24 Claude Fable 5.2 max 67.3%
  25. 25 Grok 4.20 Multi-agent low 67%
  26. 26 Gemini 3.5 Flash-Lite minimal 66%
  27. 27 Grok 4.6 minimal 66%
  28. 28 Gemini 4 Argon low 66%
  29. 29 stealth/ox-alpha low 65%
  30. 30 Kimi K2.6 none 65%
  31. 31 GPT-6.1 Sol low 65%
  32. 32 MiniMax-M3 xhigh 64%
  33. 33 Qwen3-Max high 63%
  34. 34 Muse Ember minimal 62.6%
  35. 35 MiMo-V2.5-Pro xhigh 62%
  36. 36 Claude Sonnet 5.5 low 60%
  37. 37 Kimi K3.1 Pro max 60%
  38. 38 openai/gpt-6-luna-pro max 59%
  39. 39 GPT-6 Luna none 59%
  40. 40 GPT-6 Sol none 59%
  41. 41 Claude Haiku 5.5 max 57.6%
  42. 42 Muse Wonder minimal 57.6%
  43. 43 openai/gpt-6-sol-pro max 57%
  44. 44 DeepSeek V4.1 Flash max 57%
  45. 45 Tencent Hy4 high 57%
  46. 46 Grok 4.20 low 56%
  47. 47 Grok 4.5 high 55%
  48. 48 Nemotron 3 Super xhigh 54%
  49. 49 GPT-5.6 Terra max 54%
  50. 50 GPT-6.1 Terra none 54%
  51. 51 Kimi K2.5 none 52%
  52. 52 Claude 3.5 Haiku 50%
  53. 53 Grok 4.3 minimal 50%
  54. 54 Muse Spark 1.2 minimal 50%
  55. 55 Claude 3.7 Sonnet thinking 49%
  56. 56 Nemotron 3 Ultra none 49%
  57. 57 Gemini 3 Pro low 48%
  58. 58 GPT-5.4 none 48%
  59. 59 GPT-5.6 Sol low 48%
  60. 60 GPT-5.5 xhigh 47%
  61. 61 Claude 3.5 Sonnet 46%
  62. 62 GPT-5.1 Instant 45%
  63. 63 GPT-5.2 Codex low 45%
  64. 64 openrouter/hunter-alpha none 43%
  65. 65 Claude Opus 4.1 none 43%
  66. 66 GPT-5.6 Luna max 41%
  67. 67 GPT-5.3 Instant 40%
  68. 68 GPT-5-Codex 39%
  69. 69 Gemini 3.6 Flash minimal 39%
  70. 70 DeepSeek V4 Flash 0731 xhigh 39%
  71. 71 GPT-5.2 none 38%
  72. 72 openrouter/healer-alpha none 37%
  73. 73 Gemini 3.1 Pro low 37%
  74. 74 GPT-5.5 Pro xhigh 36%
  75. 75 MiMo-V2.5 xhigh 35%
  76. 76 DeepSeek V4 Pro 0813 max 35%
  77. 77 Gemini 3.7 Flash minimal 35%
  78. 78 openai/gpt-5.5-chat 34%
  79. 79 Claude Opus 4 34%
  80. 80 GPT-5.1-Codex 32%
  81. 81 GPT-5.4 Mini high 32%
  82. 82 GLM-5-Turbo high 31%
  83. 83 GLM-5.2 xhigh 31%
  84. 84 Claude Sonnet 4 high 30%
  85. 85 Llama 4 Maverick 29%
  86. 86 Nemotron 3 Nano 30B-A3B none 28%
  87. 87 GLM-5 high 28%
  88. 88 GPT-5.2 Instant 27%
  89. 89 o3 26%
  90. 90 GPT-5.1 25%
  91. 91 Gemma 4 31B IT high 25%
  92. 92 GPT-5.3 Codex low 24%
  93. 93 Step 3.5 Flash xhigh 22%
  94. 94 GLM-5.1 xhigh 22%
  95. 95 Laguna S 2.1 none 22%
  96. 96 GPT-5 21%
  97. 97 Gemma 4 26B A4B xhigh 21%
  98. 98 Llama 4 Scout 20%
  99. 99 Gemini 2.5 Pro Jun 2025 20%
  100. 100 Qwen3-Coder-480B-A35B 20%
  101. 101 Gemini 3.5 Flash xhigh 20%
  102. 102 Gemini 2.5 Flash Sep 2025 19%
  103. 103 Grok 4.1 Fast high 19%
  104. 104 GPT-5 Chat 18%
  105. 105 DeepSeek-V4-Flash none 18%
  106. 106 Trinity Large Thinking minimal 17%
  107. 107 MiMo-V2-Flash none 16%
  108. 108 Tencent Hy3 preview none 16%
  109. 109 Gemini 2.0 Flash Feb 2025 15%
  110. 110 Llama 3.1-8B 14%
  111. 111 GPT-4.1 14%
  112. 112 GPT-5.4 Nano high 14%
  113. 113 DeepSeek-V4-Pro xhigh 14%
  114. 114 DeepSeek-V3.2 high 13%
  115. 115 GPT-4o Aug 2024 12%
  116. 116 gpt-oss-120b low 12%
  117. 117 baidu/ernie-4.5-vl-424b-a47b xhigh 11%
  118. 118 Doubao-Seed-1.6 none 11%
  119. 119 Gemini 3.1 Flash-Lite 11%
  120. 120 Claude 3 Haiku 10%
  121. 121 Kimi K2 Sep 2025 10%
  122. 122 Gemini 3 Flash high 10%
  123. 123 MiniMax-M2.5 low 9.00%
  124. 124 o4-mini low 8.00%
  125. 125 DeepSeek-R1 May 2025, xhigh 8.00%
  126. 126 GLM-4.5 xhigh 8.00%
  127. 127 MiniMax-M2.7 high 8.00%
  128. 128 INTELLECT-3 low 7.00%
  129. 129 Qwen3-235B-A22B none 6.00%
  130. 130 Mistral Small 4 high 6.00%
  131. 131 Jamba 1.7 Large 5.00%
  132. 132 nvidia-nemotron-nano-9b-v2 none 5.00%
  133. 133 deepseek/deepseek-chat 4.00%
  134. 134 ERNIE-4.5-300B-A47B 4.00%
  135. 135 Gemma 3 27B 3.00%
  136. 136 GPT-4o mini 2.00%
  137. 137 Mistral Large 3 2.00%

Source: Official BullshitBench V2 leaderboard

#EntryScoreSettingDateSource
1 Claude Opus 4.8 95% none May 2026 Official leaderboard
2 Qwen 3.8 Max 95% minimal Aug 2026 Official leaderboard
3 Claude Sonnet 4.6 91% high Feb 2026 Official leaderboard
4 Claude Opus 4.5 90% high Nov 2025 Official leaderboard
5 Claude Opus 4.6 87% high Feb 2026 Official leaderboard
6 Claude Fable 5.1 86.5% low Sep 2026 Official leaderboard
7 Claude Fable 5 83.6% xhigh Jun 2026 Official leaderboard
8 Claude Opus 4.7 83% none Apr 2026 Official leaderboard
9 Claude Sonnet 5 80.8% low Jun 2026 Official leaderboard
10 Claude Opus 5 80.2% low Jul 2026 Official leaderboard
11 Mistral Large 4 80% none Oct 2026 Official leaderboard
12 Claude Sonnet 4.5 79% high Sep 2025 Official leaderboard
13 Qwen 3.8 27B 79% none Aug 2026 Official leaderboard
14 Qwen3.5 397B-A17B 78% high Feb 2026 Official leaderboard
15 Claude Haiku 4.5 77% high Oct 2025 Official leaderboard
16 Grok 4.7 75% low Sep 2026 Official leaderboard
17 Kimi K3 74% xhigh Jul 2026 Official leaderboard
18 Qwen 3.6 Plus 72% none Mar 2026 Official leaderboard
19 Qwen3.7-Max 72% none May 2026 Official leaderboard
20 GLM-5.3 72% minimal Aug 2026 Official leaderboard
21 Gemini 4 Carbon 72% high Oct 2026 Official leaderboard
22 Claude Opus 5.5 69.2% low Sep 2026 Official leaderboard
23 GPT-6 Astra 69% max Sep 2026 Official leaderboard
24 Claude Fable 5.2 67.3% max Nov 2026 Official leaderboard
25 Grok 4.20 Multi-agent 67% low Mar 2026 Official leaderboard
26 Gemini 3.5 Flash-Lite 66% minimal Jul 2026 Official leaderboard
27 Grok 4.6 66% minimal Aug 2026 Official leaderboard
28 Gemini 4 Argon 66% low Sep 2026 Official leaderboard
29 stealth/ox-alpha 65% low Official leaderboard
30 Kimi K2.6 65% none Apr 2026 Official leaderboard
31 GPT-6.1 Sol 65% low Sep 2026 Official leaderboard
32 MiniMax-M3 64% xhigh Jun 2026 Official leaderboard
33 Qwen3-Max 63% high Sep 2025 Official leaderboard
34 Muse Ember 62.6% minimal Nov 2026 Official leaderboard
35 MiMo-V2.5-Pro 62% xhigh Apr 2026 Official leaderboard
36 Claude Sonnet 5.5 60% low Sep 2026 Official leaderboard
37 Kimi K3.1 Pro 60% max Oct 2026 Official leaderboard
38 openai/gpt-6-luna-pro 59% max Official leaderboard
39 GPT-6 Luna 59% none Sep 2026 Official leaderboard
40 GPT-6 Sol 59% none Sep 2026 Official leaderboard
41 Claude Haiku 5.5 57.6% max Oct 2026 Official leaderboard
42 Muse Wonder 57.6% minimal Oct 2026 Official leaderboard
43 openai/gpt-6-sol-pro 57% max Official leaderboard
44 DeepSeek V4.1 Flash 57% max Sep 2026 Official leaderboard
45 Tencent Hy4 57% high Nov 2026 Official leaderboard
46 Grok 4.20 56% low Feb 2026 Official leaderboard
47 Grok 4.5 55% high Jul 2026 Official leaderboard
48 Nemotron 3 Super 54% xhigh Mar 2026 Official leaderboard
49 GPT-5.6 Terra 54% max Jul 2026 Official leaderboard
50 GPT-6.1 Terra 54% none Oct 2026 Official leaderboard
51 Kimi K2.5 52% none Jan 2026 Official leaderboard
52 Claude 3.5 Haiku 50% Oct 2024 Official leaderboard
53 Grok 4.3 50% minimal Apr 2026 Official leaderboard
54 Muse Spark 1.2 50% minimal Aug 2026 Official leaderboard
55 Claude 3.7 Sonnet 49% thinking Feb 2025 Official leaderboard
56 Nemotron 3 Ultra 49% none Jun 2026 Official leaderboard
57 Gemini 3 Pro 48% low Nov 2025 Official leaderboard
58 GPT-5.4 48% none Mar 2026 Official leaderboard
59 GPT-5.6 Sol 48% low Jul 2026 Official leaderboard
60 GPT-5.5 47% xhigh Apr 2026 Official leaderboard
61 Claude 3.5 Sonnet 46% Jun 2024 Official leaderboard
62 GPT-5.1 Instant 45% Nov 2025 Official leaderboard
63 GPT-5.2 Codex 45% low Dec 2025 Official leaderboard
64 openrouter/hunter-alpha 43% none Official leaderboard
65 Claude Opus 4.1 43% none Aug 2025 Official leaderboard
66 GPT-5.6 Luna 41% max Jul 2026 Official leaderboard
67 GPT-5.3 Instant 40% Mar 2026 Official leaderboard
68 GPT-5-Codex 39% Sep 2025 Official leaderboard
69 Gemini 3.6 Flash 39% minimal Jul 2026 Official leaderboard
70 DeepSeek V4 Flash 0731 39% xhigh Jul 2026 Official leaderboard
71 GPT-5.2 38% none Dec 2025 Official leaderboard
72 openrouter/healer-alpha 37% none Official leaderboard
73 Gemini 3.1 Pro 37% low Feb 2026 Official leaderboard
74 GPT-5.5 Pro 36% xhigh Apr 2026 Official leaderboard
75 MiMo-V2.5 35% xhigh Apr 2026 Official leaderboard
76 DeepSeek V4 Pro 0813 35% max Aug 2026 Official leaderboard
77 Gemini 3.7 Flash 35% minimal Aug 2026 Official leaderboard
78 openai/gpt-5.5-chat 34% Official leaderboard
79 Claude Opus 4 34% May 2025 Official leaderboard
80 GPT-5.1-Codex 32% Nov 2025 Official leaderboard
81 GPT-5.4 Mini 32% high Mar 2026 Official leaderboard
82 GLM-5-Turbo 31% high Mar 2026 Official leaderboard
83 GLM-5.2 31% xhigh Jun 2026 Official leaderboard
84 Claude Sonnet 4 30% high May 2025 Official leaderboard
85 Llama 4 Maverick 29% Apr 2025 Official leaderboard
86 Nemotron 3 Nano 30B-A3B 28% none Dec 2025 Official leaderboard
87 GLM-5 28% high Feb 2026 Official leaderboard
88 GPT-5.2 Instant 27% Dec 2025 Official leaderboard
89 o3 26% Apr 2025 Official leaderboard
90 GPT-5.1 25% Nov 2025 Official leaderboard
91 Gemma 4 31B IT 25% high Apr 2026 Official leaderboard
92 GPT-5.3 Codex 24% low Feb 2026 Official leaderboard
93 Step 3.5 Flash 22% xhigh Feb 2026 Official leaderboard
94 GLM-5.1 22% xhigh Apr 2026 Official leaderboard
95 Laguna S 2.1 22% none Jul 2026 Official leaderboard
96 GPT-5 21% Aug 2025 Official leaderboard
97 Gemma 4 26B A4B 21% xhigh Apr 2026 Official leaderboard
98 Llama 4 Scout 20% Apr 2025 Official leaderboard
99 Gemini 2.5 Pro Jun 2025 20% Jun 2025 Official leaderboard
100 Qwen3-Coder-480B-A35B 20% Jul 2025 Official leaderboard
101 Gemini 3.5 Flash 20% xhigh May 2026 Official leaderboard
102 Gemini 2.5 Flash Sep 2025 19% Sep 2025 Official leaderboard
103 Grok 4.1 Fast 19% high Nov 2025 Official leaderboard
104 GPT-5 Chat 18% Aug 2025 Official leaderboard
105 DeepSeek-V4-Flash 18% none Apr 2026 Official leaderboard
106 Trinity Large Thinking 17% minimal Apr 2026 Official leaderboard
107 MiMo-V2-Flash 16% none Dec 2025 Official leaderboard
108 Tencent Hy3 preview 16% none Apr 2026 Official leaderboard
109 Gemini 2.0 Flash Feb 2025 15% Feb 2025 Official leaderboard
110 Llama 3.1-8B 14% Jul 2024 Official leaderboard
111 GPT-4.1 14% Apr 2025 Official leaderboard
112 GPT-5.4 Nano 14% high Mar 2026 Official leaderboard
113 DeepSeek-V4-Pro 14% xhigh Apr 2026 Official leaderboard
114 DeepSeek-V3.2 13% high Dec 2025 Official leaderboard
115 GPT-4o Aug 2024 12% Aug 2024 Official leaderboard
116 gpt-oss-120b 12% low Aug 2025 Official leaderboard
117 baidu/ernie-4.5-vl-424b-a47b 11% xhigh Official leaderboard
118 Doubao-Seed-1.6 11% none Jun 2025 Official leaderboard
119 Gemini 3.1 Flash-Lite 11% Mar 2026 Official leaderboard
120 Claude 3 Haiku 10% Mar 2024 Official leaderboard
121 Kimi K2 Sep 2025 10% Sep 2025 Official leaderboard
122 Gemini 3 Flash 10% high Dec 2025 Official leaderboard
123 MiniMax-M2.5 9.00% low Feb 2026 Official leaderboard
124 o4-mini 8.00% low Apr 2025 Official leaderboard
125 DeepSeek-R1 May 2025 8.00% xhigh May 2025 Official leaderboard
126 GLM-4.5 8.00% xhigh Aug 2025 Official leaderboard
127 MiniMax-M2.7 8.00% high Mar 2026 Official leaderboard
128 INTELLECT-3 7.00% low Nov 2025 Official leaderboard
129 Qwen3-235B-A22B 6.00% none Apr 2025 Official leaderboard
130 Mistral Small 4 6.00% high Mar 2026 Official leaderboard
131 Jamba 1.7 Large 5.00% Jul 2025 Official leaderboard
132 nvidia-nemotron-nano-9b-v2 5.00% none Aug 2025 Official leaderboard
133 deepseek/deepseek-chat 4.00% Official leaderboard
134 ERNIE-4.5-300B-A47B 4.00% Jun 2025 Official leaderboard
135 Gemma 3 27B 3.00% Mar 2025 Official leaderboard
136 GPT-4o mini 2.00% Jul 2024 Official leaderboard
137 Mistral Large 3 2.00% Dec 2025 Official leaderboard

The best score, 95%, costs $0.005 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Meta MiniMax Poolside Qwen OpenAI Mistral AI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.00001 $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Qwen 3.8 Max, minimal Mistral Large 4, none MiniMax-M3, none GPT-6 Luna, none Laguna S 2.1, none Llama 3.1-8B 0% 20% 40% 60% 80% 100% $0.00001 $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Qwen 3.8 Max, minimal Mistral Large 4, none

Source: Official BullshitBench V2 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.8 none 95% $0.02 716
Qwen 3.8 Max minimal 95% $0.005 845
Claude Opus 4.8 xhigh 94% $0.02 777
Claude Sonnet 4.6 high 91% $0.02 1,103
Claude Opus 4.5 high 90% $0.02 893
Claude Sonnet 4.6 none 89% $0.006 383
Claude Opus 4.6 high 87% $0.03 1,048
Claude Fable 5.1 low 86.5% $0.05 967
Claude Fable 5 xhigh 83.6% $0.06 1,181
Claude Opus 4.6 none 83% $0.02 597
Claude Opus 4.7 none 83% $0.02 626
Claude Sonnet 5 low 80.8% $0.008 741
Claude Opus 5 low 80.2% $0.04 1,439
Mistral Large 4 none 80% $0.001 517
Claude Sonnet 4.5 high 79% $0.01 689
Claude Opus 4.5 none 79% $0.01 371
Qwen 3.8 27B none 79% $0.002 929
Claude Sonnet 5 max 78.8% $0.11 10,830
Claude Opus 5 xhigh 78.7% $0.08 3,315
Qwen3.5 397B-A17B high 78% $0.009 2,484
Claude Haiku 4.5 high 77% $0.005 900
Grok 4.7 low 75% $0.005 638
Claude Fable 5.1 max 74.4% $0.24 4,837
Claude Sonnet 4.5 none 74% $0.006 378
Claude Opus 4.7 max 74% $0.05 2,175
Kimi K3 xhigh 74% $0.04 2,759
Qwen 3.6 Plus none 72% $0.002 827
Qwen3.7-Max none 72% $0.006 775
Kimi K3 minimal 72% $0.04 2,837
GLM-5.3 minimal 72% $0.002 517
Grok 4.7 xhigh 72% $0.007 962
Gemini 4 Carbon high 72% $0.04 3,870
Claude Haiku 4.5 none 71% $0.002 363
Claude Fable 5 low 70.3% $0.06 1,188
Claude Opus 5.5 low 69.2% $0.02 1,086
Qwen3.5 397B-A17B none 69% $0.003 817
GPT-6 Astra max 69% $0.16 3,095
Claude Opus 5.5 max 68.9% $0.31 15,689
Mistral Large 4 high 68% $0.003 1,562
Claude Fable 5.2 max 67.3% $0.12 3,006
Grok 4.20 Multi-agent low 67% $0.09 12,787
Gemini 3.5 Flash-Lite minimal 66% $0.001 514
Grok 4.6 minimal 66% $0.004 674
GLM-5.3 max 66% $0.02 4,402
Gemini 4 Argon low 66% $0.008 831
Kimi K2.6 none 65% $0.002 549
GPT-6.1 Sol low 65% $0.004 423
Gemini 4 Argon high 65% $0.02 2,021
Grok 4.20 Multi-agent xhigh 64% $0.41 55,511
MiniMax-M3 xhigh 64% $0.002 1,405
GPT-6 Astra low 64% $0.02 415
Claude Fable 5.2 low 63.3% $0.05 1,140
Qwen3-Max high 63% $0.02 4,285
MiniMax-M3 none 63% $0.0007 545
Muse Ember minimal 62.6% $0.0005 1,024
MiMo-V2.5-Pro xhigh 62% $0.003 928
Gemini 4 Carbon low 61% $0.01 903
Gemini 3.5 Flash-Lite xhigh 60% $0.004 1,595
Claude Sonnet 5.5 low 60% $0.01 957
Kimi K3.1 Pro max 60% $0.03 2,804
Qwen 3.6 Plus xhigh 59% $0.004 2,269
GPT-6 Luna none 59% $0.0001 216
GPT-6 Sol none 59% $0.002 194
GPT-6.1 Sol max 59% $0.03 2,733
GPT-6 Luna max 58% $0.002 4,469
Claude Sonnet 5.5 max 58% $0.08 7,634
Claude Haiku 5.5 max 57.6% $0.01 22,447
Muse Wonder minimal 57.6% $0.02 993
Muse Ember max 57.6% $0.001 2,806
Qwen3.7-Max xhigh 57% $0.02 2,218
DeepSeek V4.1 Flash max 57% $0.005 5,035
DeepSeek V4.1 Flash low 57% $0.003 2,522
Tencent Hy4 high 57% $0.007 5,952
Grok 4.20 low 56% $0.01 1,946
Grok 4.6 max 56% $0.01 1,641
Grok 4.5 high 55% $0.01 1,540
Grok 4.20 xhigh 54% $0.01 1,992
Nemotron 3 Super xhigh 54% 2,236
Grok 4.5 low 54% $0.006 888
GPT-5.6 Terra max 54% $0.08 5,031
GPT-6 Sol max 54% $0.02 1,694
GPT-6.1 Terra none 54% $0.001 189
Tencent Hy4 none 54% $0.001 849
Claude Haiku 5.5 low 53% $0.0006 1,197
GPT-6.1 Terra max 53% $0.03 5,854
Muse Wonder max 52.5% $0.06 2,977
Kimi K2.5 none 52% $0.001 407
Claude 3.5 Haiku 50% $0.0006 138
Kimi K2.6 xhigh 50% $0.01 2,835
Grok 4.3 minimal 50% $0.006 2,249
Muse Spark 1.2 minimal 50% $0.005 1,055
Kimi K3.1 Pro none 50% $0.01 944
Claude 3.7 Sonnet thinking 49% $0.01 755
Nemotron 3 Ultra none 49% $0.002 789
Muse Spark 1.2 xhigh 49% $0.008 1,776
Gemini 3 Pro low 48% $0.01 928
GPT-5.4 none 48% $0.009 615
GPT-5.6 Sol low 48% $0.02 493
Nemotron 3 Super high 47% $0.001 1,811
GPT-5.5 xhigh 47% 2,491
GPT-5.6 Sol max 47% $0.07 2,448
GPT-5.6 Terra low 47% $0.01 766
Claude 3.5 Sonnet 46% $0.004 115
Qwen3-Max none 46% $0.003 670
Grok 4.3 xhigh 46% $0.005 2,053
GPT-5.1 Instant 45% $0.003 270
GPT-5.2 Codex low 45% $0.005 336
GPT-5.5 none 45% 634
GPT-5.5 low 45% 836
Claude 3.7 Sonnet 44% $0.004 242
Claude Opus 4.1 none 43% $0.03 324
Nemotron 3 Super none 43% 1,440
Claude Opus 4.1 high 42% $0.05 619
GPT-5.4 xhigh 42% $0.10 6,706
GPT-5.6 Luna max 41% $0.03 5,353
GPT-5.3 Instant 40% $0.007 471
Nemotron 3 Ultra xhigh 40% $0.003 1,301
GPT-5-Codex 39% $0.01 994
GPT-5.2 Codex xhigh 39% $0.05 3,578
Gemini 3.6 Flash minimal 39% $0.008 1,097
DeepSeek V4 Flash 0731 xhigh 39% $0.001 4,260
GPT-5.2 none 38% $0.01 974
MiMo-V2.5-Pro none 38% $0.001 418
GPT-5.2 Codex high 37% $0.03 2,104
Gemini 3.1 Pro low 37% $0.02 1,449
GPT-5.6 Luna low 37% $0.004 622
Gemini 3 Pro high 36% $0.03 2,248
GPT-5.5 Pro xhigh 36% 5,322
MiMo-V2.5 xhigh 35% $0.002 936
DeepSeek V4 Pro 0813 max 35% $0.008 4,424
Gemini 3.7 Flash minimal 35% $0.002 1,096
Claude Opus 4 34% $0.02 311
GPT-5.5 Pro medium 34% 4,180
GPT-5.1-Codex 32% $0.005 446
GPT-5.4 Mini high 32% $0.006 1,368
GPT-5.4 Mini none 32% $0.002 477
DeepSeek V4 Pro 0813 low 32% $0.004 2,384
Kimi K2.5 high 31% $0.006 2,038
Gemini 3.1 Pro high 31% $0.03 2,231
Nemotron 3 Super none 31% $0.0006 958
GLM-5-Turbo high 31% $0.01 2,481
GPT-5.4 Mini xhigh 31% $0.03 7,762
GLM-5.2 xhigh 31% $0.01 2,515
Gemini 3.7 Flash max 31% $0.004 2,351
Claude Sonnet 4 high 30% $0.01 718
Llama 4 Maverick 29% $0.0004 522
Claude Sonnet 4 none 29% $0.005 350
Qwen 3.8 27B max 29% $0.02 8,769
GPT-5.2 high 28% $0.02 1,715
Nemotron 3 Nano 30B-A3B none 28% 1,672
GLM-5 high 28% $0.009 2,903
Gemini 3.6 Flash xhigh 28% $0.02 2,493
GPT-5.2 Instant 27% $0.01 824
Qwen 3.8 Max max 27% $0.04 6,224
o3 26% $0.01 1,368
GPT-5.1 25% $0.02 1,894
Gemma 4 31B IT high 25% $0.0006 1,540
GPT-5.3 Codex low 24% $0.008 552
MiMo-V2.5 none 24% $0.001 580
GLM-5-Turbo none 23% $0.005 1,130
Step 3.5 Flash xhigh 22% $0.001 4,349
GLM-5.1 xhigh 22% $0.01 2,611
Laguna S 2.1 none 22% $0.0001 503
GPT-5 21% $0.03 2,918
Gemma 4 26B A4B xhigh 21% $0.0008 1,868
Laguna S 2.1 xhigh 21% $0.0002 831
Llama 4 Scout 20% $0.0002 517
Gemini 2.5 Pro Jun 2025 20% $0.03 3,431
Qwen3-Coder-480B-A35B 20% $0.0005 288
GPT-5.3 Codex high 20% $0.02 1,197
GLM-5 none 20% $0.003 984
Gemma 4 31B IT none 20% $0.0003 803
Gemini 3.5 Flash xhigh 20% $0.03 2,828
DeepSeek V4 Flash 0731 none 20% $0.0003 1,152
Gemini 2.5 Flash Sep 2025 19% $0.005 1,927
Grok 4.1 Fast high 19% $0.0008 1,599
GPT-5.3 Codex xhigh 19% $0.04 2,685
Gemini 3.5 Flash minimal 19% $0.01 1,240
GPT-5 Chat 18% $0.007 655
GLM-5.1 none 18% $0.005 1,128
DeepSeek-V4-Flash none 18% $0.0003 1,154
Trinity Large Thinking minimal 17% $0.002 2,722
GLM-5.2 none 17% $0.005 1,030
MiMo-V2-Flash none 16% $0.0002 692
Tencent Hy3 preview none 16% 2,480
Gemini 2.0 Flash Feb 2025 15% $0.0006 1,585
Llama 3.1-8B 14% $0.00003 528
GPT-4.1 14% $0.006 764
GPT-5.4 Nano high 14% $0.002 1,818
GPT-5.4 Nano none 14% $0.001 860
DeepSeek-V4-Flash xhigh 14% $0.002 8,141
DeepSeek-V4-Pro xhigh 14% $0.01 3,814
DeepSeek-V4-Pro none 14% $0.003 978
DeepSeek-V3.2 high 13% $0.0008 1,587
MiMo-V2-Flash high 13% $0.0009 2,845
Step 3.5 Flash minimal 13% $0.001 4,446
Trinity Large Thinking xhigh 13% $0.003 3,070
GPT-4o Aug 2024 12% $0.004 392
gpt-oss-120b low 12% $0.0009 2,293
Doubao-Seed-1.6 none 11% $0.003 1,648
Gemini 3.1 Flash-Lite 11% $0.001 914
Gemma 4 26B A4B none 11% $0.0004 923
Claude 3 Haiku 10% $0.0004 339
Kimi K2 Sep 2025 10% $0.001 615
Grok 4.1 Fast none 10% $0.0005 955
DeepSeek-V3.2 none 10% $0.0004 943
Nemotron 3 Nano 30B-A3B xhigh 10% 3,175
Gemini 3 Flash high 10% $0.005 1,714
Gemini 3 Flash none 10% $0.003 870
GPT-5.4 Nano xhigh 10% $0.008 6,754
MiniMax-M2.5 low 9.00% $0.004 2,911
o4-mini low 8.00% $0.004 906
DeepSeek-R1 May 2025 xhigh 8.00% $0.004 1,692
GLM-4.5 xhigh 8.00% $0.006 2,632
MiniMax-M2.5 high 8.00% $0.003 2,241
MiniMax-M2.7 high 8.00% $0.005 2,258
Tencent Hy3 preview xhigh 8.00% 4,521
DeepSeek-R1 May 2025 none 7.00% $0.004 1,652
Doubao-Seed-1.6 high 7.00% $0.006 3,109
INTELLECT-3 low 7.00% $0.004 3,279
MiniMax-M2.7 low 7.00% $0.005 2,094
Qwen3-235B-A22B none 6.00% $0.002 1,022
GLM-4.5 none 6.00% $0.005 2,319
Mistral Small 4 high 6.00% $0.001 2,039
Jamba 1.7 Large 5.00% $0.007 798
gpt-oss-120b high 5.00% $0.002 4,679
nvidia-nemotron-nano-9b-v2 none 5.00% 1,863
INTELLECT-3 high 5.00% $0.004 3,215
o4-mini high 4.00% $0.01 2,707
ERNIE-4.5-300B-A47B 4.00% $0.001 1,030
Mistral Small 4 none 4.00% $0.0006 1,020
Gemma 3 27B 3.00% $0.0005 1,730
Qwen3-235B-A22B xhigh 3.00% $0.005 2,539
nvidia-nemotron-nano-9b-v2 xhigh 3.00% 1,727
GPT-4o mini 2.00% $0.0003 472
Mistral Large 3 2.00% $0.003 1,707

2 models have held number one on the official leaderboard since its release in March 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%

Source: Official BullshitBench V2 leaderboard

Claude Opus 4.5 on the official leaderboard passed 85% in November 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Claude Opus 4.8 Claude Opus 4.5 Claude Sonnet 4.5 Qwen3-Max Claude 3.5 Haiku Claude 3 Haiku 0% 20% 40% 60% 80% 100% 2024 2025 2026 2027 Near the ceiling (85%) Claude Opus 4.8 Claude Sonnet 4.5

Source: Official BullshitBench V2 leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic
BullshitBench V2 Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%
BullshitBench Passed 85% in Feb 2026, by Claude Opus 4.5
Claude Opus 4.8 96.4%
BullshitBench V2 Passed 85% in Mar 2026, by Claude Sonnet 4.6
Claude Opus 4.8 95%

Source: Official leaderboards

Replaces BullshitBench

More in Factuality

All 74 in Factuality
  • SimpleVQA In use Models answer 2,025 short factual questions about images across 9 tasks and 9 topics. An LLM judge grades answers against fixed references. Qwen3.7-Plus 81.7%
  • HallusionBench In use Models answer 1,129 yes or no questions about 346 images, built in pairs to expose language hallucination and visual illusion. Scored by accuracy. Keye-VL-2.0-30B-A3B 73.5%
  • SimpleQA Verified In use 1,000 human-verified, de-duplicated short-answer factual questions measuring parametric knowledge. Gemini 3.1 Pro 77.5%
  • Kimi Hallu Bench In use Models answer hard prompts drawn from practical work tasks. Fact-checking agents score the share of claims that are hallucinated. GPT-6 Astra 4.20%
  • Internal adversarially-designed factuality evaluations In use Models answer adversarial questions in nine categories such as history, math and code. Scored on agreement with human ideal responses. GPT-6 Luna 7.60%
  • AA-Omniscience v1.1 In use Models answer 3,000 knowledge questions in six domains without tools and may abstain. The index adds correct answers and subtracts wrong ones. Inkling 2 Small 16.1