Benchmark Almanac

HalachaBench

Claude Fable 5.2 85.8 leads since November 2026
Not in model cards

Models answer 60 Hebrew rabbinical qualification exam questions. A model judge awards rubric credit on a scale from 0 to 100.

Each model answers every question three times. AI generated the rubrics and judged the answers. No rabbi reviewed the evaluation, and scores are not qualification grades. The public result bundle withholds question text and rubric text.

Made by
HalachaBench
Released
November 2026
Kind
Other professions
Level
Expert
Scored
Graded by a model
Answers
Open-ended
Tasks
Kept private
Measures
Capability
Language
Hebrew
In and out
Text → Text
Interaction
Single turn
Items from
Exams and contests
Refresh
One fixed set
Run by
A host
Occupations
Community and social service
In views
Law and finance
Strongest model tested
Claude Fable 5.2 #2 on the capability index
Last model added
November 2026
Scores refreshed
November 18th, 2026
30
Entries on its own leaderboard, 28 models we track

Claude Fable 5.2 leads with 85.8 Claude Fable 5.2 leads with 85.8

Best score per entry from the benchmark's own leaderboard, read on November 18th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

  1. 1 Claude Fable 5.2 provisional; 60 test questions; effort high 85.8
  2. 2 Gemini 4 Carbon provisional; 60 test questions; effort high 82.3
  3. 3 Claude Fable 5.1 provisional; 60 test questions; effort high 79.7
  4. 4 Gemini 4 Argon provisional; 60 test questions; effort high 78.2
  5. 5 GPT-5.6 Sol provisional; 60 test questions; effort high 77.3
  6. 6 GPT-6 Astra provisional; 60 test questions; effort high 73.3
  7. 7 GPT-6.1 Sol provisional; 60 test questions; effort high 72.8
  8. 8 Claude Opus 5.5 provisional; 60 test questions; effort high 69.1
  9. 9 Gemini 3.8 Flash provisional; 60 test questions; effort high 64.4
  10. 10 Muse Wonder Full provisional; 60 test questions; effort high 60.8
  11. 11 GPT-6 Sol provisional; 60 test questions; effort high 48.7
  12. 12 Muse Wonder provisional; 60 test questions; effort high 48.2
  13. 13 Claude Opus 4.8 provisional; 60 test questions; effort high 48.0
  14. 14 Muse Spark 1.3 provisional; 60 test questions; effort high 36.2
  15. 15 GPT-6.1 Terra provisional; 60 test questions; effort high 34.6
  16. 16 GPT-6 Luna provisional; 60 test questions; effort high 32.4
  17. 17 Grok 4.7 provisional; 60 test questions; effort high 29.9
  18. 18 Claude Sonnet 4.6 provisional; 60 test questions; effort high 29.6
  19. 19 Claude Sonnet 5.5 provisional; 60 test questions; effort high 28.7
  20. 20 Muse Ember provisional; 60 test questions; effort high 15.3
  21. 21 Grok 4.3 provisional; 60 test questions; effort high 10.8
  22. 22 Muse Glimmer provisional; 60 test questions; llama.cpp; Q4_K_M 10.5
  23. 23 Claude Haiku 5.5 provisional; 60 test questions; effort high 8.60
  24. 24 Ling-3.0-flash provisional; 60 test questions; llama.cpp; Q4_K_M 6.10
  25. 25 Qwen 3.8 27B provisional; 60 test questions; llama.cpp; UD-Q4_K_M 6.00
  26. 26 gpt-oss-120b provisional; 60 test questions; llama.cpp; MXFP4 4.40
  27. 27 DictaLM 3.0 24B Thinking provisional; 60 test questions; llama.cpp; Q4_K_M 4.30
  28. 28 Gemma 4 31B IT provisional; 60 test questions; llama.cpp; Q4_K_M 4.30
  29. 29 Llama 4 Scout 17B-16E provisional; 60 test questions; llama.cpp; Q4_K_M 4.20
  30. 30 gpt-oss-20b provisional; 60 test questions; llama.cpp; MXFP4 1.20

Source: Official HalachaBench leaderboard

  1. 1 Claude Fable 5.2 provisional; 60 test questions; effort high 85.8
  2. 2 Gemini 4 Carbon provisional; 60 test questions; effort high 82.3
  3. 3 Claude Fable 5.1 provisional; 60 test questions; effort high 79.7
  4. 4 Gemini 4 Argon provisional; 60 test questions; effort high 78.2
  5. 5 GPT-5.6 Sol provisional; 60 test questions; effort high 77.3
  6. 6 GPT-6 Astra provisional; 60 test questions; effort high 73.3
  7. 7 GPT-6.1 Sol provisional; 60 test questions; effort high 72.8
  8. 8 Claude Opus 5.5 provisional; 60 test questions; effort high 69.1
  9. 9 Gemini 3.8 Flash provisional; 60 test questions; effort high 64.4
  10. 10 Muse Wonder Full provisional; 60 test questions; effort high 60.8
  11. 11 GPT-6 Sol provisional; 60 test questions; effort high 48.7
  12. 12 Muse Wonder provisional; 60 test questions; effort high 48.2
  13. 13 Claude Opus 4.8 provisional; 60 test questions; effort high 48.0
  14. 14 Muse Spark 1.3 provisional; 60 test questions; effort high 36.2
  15. 15 GPT-6.1 Terra provisional; 60 test questions; effort high 34.6
  16. 16 GPT-6 Luna provisional; 60 test questions; effort high 32.4
  17. 17 Grok 4.7 provisional; 60 test questions; effort high 29.9
  18. 18 Claude Sonnet 4.6 provisional; 60 test questions; effort high 29.6
  19. 19 Claude Sonnet 5.5 provisional; 60 test questions; effort high 28.7
  20. 20 Muse Ember provisional; 60 test questions; effort high 15.3
  21. 21 Grok 4.3 provisional; 60 test questions; effort high 10.8
  22. 22 Muse Glimmer provisional; 60 test questions; llama.cpp; Q4_K_M 10.5
  23. 23 Claude Haiku 5.5 provisional; 60 test questions; effort high 8.60
  24. 24 Ling-3.0-flash provisional; 60 test questions; llama.cpp; Q4_K_M 6.10
  25. 25 Qwen 3.8 27B provisional; 60 test questions; llama.cpp; UD-Q4_K_M 6.00
  26. 26 gpt-oss-120b provisional; 60 test questions; llama.cpp; MXFP4 4.40
  27. 27 Gemma 4 31B IT provisional; 60 test questions; llama.cpp; Q4_K_M 4.30
  28. 28 gpt-oss-20b provisional; 60 test questions; llama.cpp; MXFP4 1.20

Source: Official HalachaBench leaderboard

#EntryScore ±SettingDateSource
1 Claude Fable 5.2 85.8 82.7 to 88.5 provisional; 60 test questions; effort high, Claude Code CLI Nov 2026 Official leaderboard
2 Gemini 4 Carbon 82.3 78.4 to 86 provisional; 60 test questions; effort high, Antigravity CLI Oct 2026 Official leaderboard
3 Claude Fable 5.1 79.7 75.8 to 83.4 provisional; 60 test questions; effort high, Claude Code CLI Sep 2026 Official leaderboard
4 Gemini 4 Argon 78.2 74.3 to 81.9 provisional; 60 test questions; effort high, Antigravity CLI Sep 2026 Official leaderboard
5 GPT-5.6 Sol 77.3 72.4 to 81.3 provisional; 60 test questions; effort high, Codex Jul 2026 Official leaderboard
6 GPT-6 Astra 73.3 69.4 to 77 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
7 GPT-6.1 Sol 72.8 69.1 to 76.2 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
8 Claude Opus 5.5 69.1 64.7 to 73.3 provisional; 60 test questions; effort high, Claude Code CLI Sep 2026 Official leaderboard
9 Gemini 3.8 Flash 64.4 59.7 to 68.9 provisional; 60 test questions; effort high, Antigravity CLI Sep 2026 Official leaderboard
10 Muse Wonder Full 60.8 ±4.4 provisional; 60 test questions; effort high, Muse Code CLI Oct 2026 Official leaderboard
11 GPT-6 Sol 48.7 ±4.6 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
12 Muse Wonder 48.2 ±4.9 provisional; 60 test questions; effort high, Muse Code CLI Oct 2026 Official leaderboard
13 Claude Opus 4.8 48.0 ±4.7 provisional; 60 test questions; effort high, Claude Code CLI May 2026 Official leaderboard
14 Muse Spark 1.3 36.2 32.2 to 40.4 provisional; 60 test questions; effort high, Muse Code CLI Sep 2026 Official leaderboard
15 GPT-6.1 Terra 34.6 ±3.9 provisional; 60 test questions; effort high, Codex Oct 2026 Official leaderboard
16 GPT-6 Luna 32.4 ±3.9 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
17 Grok 4.7 29.9 ±3.7 provisional; 60 test questions; effort high, xAI API Sep 2026 Official leaderboard
18 Claude Sonnet 4.6 29.6 ±4.3 provisional; 60 test questions; effort high, Claude Code CLI Feb 2026 Official leaderboard
19 Claude Sonnet 5.5 28.7 ±4.5 provisional; 60 test questions; effort high, Claude Code CLI Sep 2026 Official leaderboard
20 Muse Ember 15.3 12 to 18.8 provisional; 60 test questions; effort high, Muse Code CLI Nov 2026 Official leaderboard
21 Grok 4.3 10.8 ±2.8 provisional; 60 test questions; effort high, xAI API Apr 2026 Official leaderboard
22 Muse Glimmer 10.5 7.6 to 13.7 provisional; 60 test questions; llama.cpp; Q4_K_M, local Aug 2026 Official leaderboard
23 Claude Haiku 5.5 8.60 6 to 11.5 provisional; 60 test questions; effort high, Claude Code CLI Oct 2026 Official leaderboard
24 Ling-3.0-flash 6.10 4.4 to 8 provisional; 60 test questions; llama.cpp; Q4_K_M, local Aug 2026 Official leaderboard
25 Qwen 3.8 27B 6.00 4.1 to 8.2 provisional; 60 test questions; llama.cpp; UD-Q4_K_M, local Aug 2026 Official leaderboard
26 gpt-oss-120b 4.40 2.6 to 6.4 provisional; 60 test questions; llama.cpp; MXFP4, local Aug 2025 Official leaderboard
27 DictaLM 3.0 24B Thinking 4.30 2.8 to 6 provisional; 60 test questions; llama.cpp; Q4_K_M, local Official leaderboard
28 Gemma 4 31B IT 4.30 2.5 to 6.4 provisional; 60 test questions; llama.cpp; Q4_K_M, local Apr 2026 Official leaderboard
29 Llama 4 Scout 17B-16E 4.20 2.6 to 6 provisional; 60 test questions; llama.cpp; Q4_K_M, local Official leaderboard
30 gpt-oss-20b 1.20 0.5 to 2.1 provisional; 60 test questions; llama.cpp; MXFP4, local Aug 2025 Official leaderboard
#ModelScore ±SettingDateSource
1 Claude Fable 5.2 85.8 82.7 to 88.5 provisional; 60 test questions; effort high, Claude Code CLI Nov 2026 Official leaderboard
2 Gemini 4 Carbon 82.3 78.4 to 86 provisional; 60 test questions; effort high, Antigravity CLI Oct 2026 Official leaderboard
3 Claude Fable 5.1 79.7 75.8 to 83.4 provisional; 60 test questions; effort high, Claude Code CLI Sep 2026 Official leaderboard
4 Gemini 4 Argon 78.2 74.3 to 81.9 provisional; 60 test questions; effort high, Antigravity CLI Sep 2026 Official leaderboard
5 GPT-5.6 Sol 77.3 72.4 to 81.3 provisional; 60 test questions; effort high, Codex Jul 2026 Official leaderboard
6 GPT-6 Astra 73.3 69.4 to 77 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
7 GPT-6.1 Sol 72.8 69.1 to 76.2 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
8 Claude Opus 5.5 69.1 64.7 to 73.3 provisional; 60 test questions; effort high, Claude Code CLI Sep 2026 Official leaderboard
9 Gemini 3.8 Flash 64.4 59.7 to 68.9 provisional; 60 test questions; effort high, Antigravity CLI Sep 2026 Official leaderboard
10 Muse Wonder Full 60.8 ±4.4 provisional; 60 test questions; effort high, Muse Code CLI Oct 2026 Official leaderboard
11 GPT-6 Sol 48.7 ±4.6 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
12 Muse Wonder 48.2 ±4.9 provisional; 60 test questions; effort high, Muse Code CLI Oct 2026 Official leaderboard
13 Claude Opus 4.8 48.0 ±4.7 provisional; 60 test questions; effort high, Claude Code CLI May 2026 Official leaderboard
14 Muse Spark 1.3 36.2 32.2 to 40.4 provisional; 60 test questions; effort high, Muse Code CLI Sep 2026 Official leaderboard
15 GPT-6.1 Terra 34.6 ±3.9 provisional; 60 test questions; effort high, Codex Oct 2026 Official leaderboard
16 GPT-6 Luna 32.4 ±3.9 provisional; 60 test questions; effort high, Codex Sep 2026 Official leaderboard
17 Grok 4.7 29.9 ±3.7 provisional; 60 test questions; effort high, xAI API Sep 2026 Official leaderboard
18 Claude Sonnet 4.6 29.6 ±4.3 provisional; 60 test questions; effort high, Claude Code CLI Feb 2026 Official leaderboard
19 Claude Sonnet 5.5 28.7 ±4.5 provisional; 60 test questions; effort high, Claude Code CLI Sep 2026 Official leaderboard
20 Muse Ember 15.3 12 to 18.8 provisional; 60 test questions; effort high, Muse Code CLI Nov 2026 Official leaderboard
21 Grok 4.3 10.8 ±2.8 provisional; 60 test questions; effort high, xAI API Apr 2026 Official leaderboard
22 Muse Glimmer 10.5 7.6 to 13.7 provisional; 60 test questions; llama.cpp; Q4_K_M, local Aug 2026 Official leaderboard
23 Claude Haiku 5.5 8.60 6 to 11.5 provisional; 60 test questions; effort high, Claude Code CLI Oct 2026 Official leaderboard
24 Ling-3.0-flash 6.10 4.4 to 8 provisional; 60 test questions; llama.cpp; Q4_K_M, local Aug 2026 Official leaderboard
25 Qwen 3.8 27B 6.00 4.1 to 8.2 provisional; 60 test questions; llama.cpp; UD-Q4_K_M, local Aug 2026 Official leaderboard
26 gpt-oss-120b 4.40 2.6 to 6.4 provisional; 60 test questions; llama.cpp; MXFP4, local Aug 2025 Official leaderboard
27 Gemma 4 31B IT 4.30 2.5 to 6.4 provisional; 60 test questions; llama.cpp; Q4_K_M, local Apr 2026 Official leaderboard
28 gpt-oss-20b 1.20 0.5 to 2.1 provisional; 60 test questions; llama.cpp; MXFP4, local Aug 2025 Official leaderboard

2 models have held number one on the official leaderboard since its release in November 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Anthropic
Own leaderboard 2 leaders

Source: Official HalachaBench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 20.0 40.0 60.0 80.0 100.0 2026 2027 0.00 20.0 40.0 60.0 80.0 100.0 2026 2027

Source: Official HalachaBench leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 637 in Law, finance and medicine
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • FinSearchComp In use Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. Ling-3.1-flash 81.7%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. o1-preview 92.7%
  • Harvey LAB-AA v1.1 In use Artificial Analysis's run of Harvey's Legal Agent Benchmark on Harvey's newer private set of 120 tasks across 24 practice areas. A deliverable with a material hallucination scores zero, and the headline is the share of judges who find every rubric criterion passed. GPT-6.2 Astra 15%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access The tasks are private.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (30 entries).
  • Pass Models scored 28 models we track, plus 2 other entries on its leaderboard.
  • Pass Newest score The newest is from November 2026, for Claude Fable 5.2.
  • Pass Frontier models 9 of the top 10 have a score, led by Claude Fable 5.2 (#2).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on github.com: 28 rows for models we track and 2 other entries at the last read.
  • Pass Last read November 18th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check