Benchmark Almanac

Decision Index Public v0.3

Decision models return probabilities over labeled options in public tasks covering knowledge, language, retrieval, tools, and arts. A chance-corrected weighted aggregate combines the task scores.

This is the public v0.3 component. The full Decision Index uses separate private tests, so its scores are not interchangeable. RouterBench and SGD appear as diagnostics but do not count toward this aggregate. Release month remains unconfirmed.

Kind
Composite indices
Level
University
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Items from
Another benchmark
Run by
Anyone
Based on
Decision Index
Combines
An average of other benchmarks
Last in a launch post
October 2026
Last model added
October 2026
2
Models scored in lab cards
2
Launch posts list it, from 1 lab

Granite Switch 4.1 30B leads with 51.3

Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 1 Granite Switch 4.1 30B 51.3
  2. 2 Granite Switch 4.1 8B 46.7

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 Granite Switch 4.1 30B 51.3 Oct 2026 Lab model card
2 Granite Switch 4.1 8B 46.7 Oct 2026 Lab model card

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card
46 48 50 52 2027 46 48 50 52 2027

Source: Labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

IBM
DateLaunch postScoreAs the post writes it
Oct 27, 2026 Granite Switch 4.1 8B 46.7 Decision Index (JEV) — decision aLoRA, Overall
Oct 27, 2026 Granite Switch 4.1 30B 51.3 Decision Index (JEV) — decision aLoRA, Overall

More in Other

All 69 in Other
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Claude Fable 5.2 64.6
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Vals Science Index In use Agents are scored on the equal average of BioMysteryBench, MysteryMechanism and Terminal-Bench-Science. GPT-6.2 Astra 68.4%
  • SAGE v1.1 (Vals) In use Models grade handwritten undergraduate math and computer science work against rubrics. Scored by class-balanced accuracy of each rubric check against expert grading. Claude Opus 5.5 91.3%
  • MTEB Fading Massive Text Embedding Benchmark: retrieval, clustering, classification across languages and modalities. Qwen3-Embedding-8B 72.2%
  • SPEED-Bench Only in system cards Measures speculative decoding across varied prompts and domains. Reports the average acceptance length of draft tokens, so higher means faster generation at the same output. Qwen3-Next-80B-A3B-Instruct 2.81

Data audit

All 4 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 pass 4 notes
  • Note Independent runs Only labs’ own numbers, so no one else has checked them. Scores from launch posts (2 scores).
  • Note Models scored Only 2 models we track.
  • Pass Newest score The newest is from October 2026, for Granite Switch 4.1 30B.
  • Note Frontier models None of its models are on the capabilities index.
  • Note Headroom Scored as an index, with no fixed ceiling.
Leaderboard import 1 note
  • Note Importer No public leaderboard to read.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check