Cohere Command A
Cohere Command A ranks first on 2 of the 42 benchmarks with five or more models scored.
59
Benchmarks with a score
2
Ranked first, out of five or more models
3
In the top three
Cohere Mar 2025 Cohere Command A
Open weights
122.9
#330 of 404
Self-Reported Index, from the lab's own numbers. No Epoch Capabilities Index score yet.
- LongBench v2 43.4% #31 /35
- Vals Finance Agent 4.23% #47 /49
- LiveCodeBench v5 26.9% #45 /47
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| RULER | 95% | #6 of 42 | Lab model card | |
| DROP | 91.1% | #6 of 41 | Lab model card | |
| IFEval | 90.9% | #25 of 135 | Lab model card | |
| MGSM | 90.1% | #12 of 57 | Lab model card | |
| RepoQA | 92.6% | #1 of 4 | Lab model card | |
| MMLU | 85.5% | #51 of 131 | Lab model card | |
| Vectara Hallucination Leaderboard (2025 dataset) | 9.30% | #47 of 104 | Official leaderboard | |
| LMArena (Elo) | 1,354 | #173 of 335 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.50% | #78 of 148 | Official leaderboard | |
| Wer wird Millionär? LLM Benchmark | 2800.0 | #41 of 77 | Official leaderboard | |
| INCLUDE | 74.3% | #17 of 31 | Lab model card | |
| HELM Arabic | 77.7 | #17 of 28 | Official leaderboard | |
| Global MMLU Lite | 83.8% | #29 of 43 | Official leaderboard | |
| MMLU-Pro | 69.6% | #122 of 167 | Lab model card | |
| Phare | 60.9% | #49 of 62 | Official leaderboard | |
| HUMAINE | 20 | #42 of 50 | Official leaderboard | |
| LongBench v2 | 43.4% | #31 of 35 | Lab model card | |
| HELM Arabic Enterprise | 58.0 | #21 of 23 | Official leaderboard | |
| StrategyQA | 76.7% | #2 of 2 | Lab model card | |
| Bamboogle | 76% | #1 of 1 | Lab model card | |
| Enterprise RAG evaluation (Cohere) | 4.73 | #1 of 1 | Lab model card | |
| FLORES-200 | 81.2 | #1 of 1 | Lab model card | |
| HotpotQA | 92.1% | #1 of 1 | Lab model card | |
| InFoBench | 94.9% | #1 of 1 | Lab model card | |
| Language Confusion Benchmark | 93% | #1 of 1 | Lab model card | |
| NTREX-128 | 68.8 | #1 of 1 | Lab model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 80% | #28 of 99 | Lab model card | |
| DTBench | 61.3% | #139 of 180 | Official leaderboard | |
| Misguided Attention | 30.1% | #35 of 45 | Official leaderboard | |
| GPQA Diamond | 50.8% | #243 of 302 | Lab model card | |
| AIME 2024 | 23.3% | #84 of 88 | Lab model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP+ | 86.2% | #1 of 15 | Lab model card | |
| Bird-SQL | 59.5% | #3 of 13 | Lab model card | |
| HumanEvalPack | 76.2% | #1 of 3 | Lab model card | |
| HumanEval | 85.4% | #39 of 101 | Lab model card | |
| Spider | 80.2% | #2 of 4 | Lab model card | |
| BigCodeBench | 45.4% | #11 of 14 | Lab model card | |
| LiveCodeBench v5 | 26.9% | #45 of 47 | Lab model card | |
| Aider Polyglot | 14.7% | #29 of 30 | Lab model card | |
| SWE-bench Verified | 26.8% | #151 of 155 | Lab model card | |
| LBPP (Less Basic Python Problems) | 51.5% | #2 of 2 | Lab model card | |
| HumanEval-COBOL (Cohere) | 25.3% | #1 of 1 | Lab model card | |
| Multi-dialect SQL evaluation (Cohere) | 55.3% | #1 of 1 | Lab model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BFCL V4 | 46.5% | #24 of 65 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 63.8% | #33 of 41 | Lab model card | |
| tau-bench | 51.7% | #21 of 23 | Lab model card |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CaseLaw v2 | 64.5% | #13 of 53 | Official leaderboard | |
| LegalBench | 79.7% | #88 of 138 | Official leaderboard | |
| TaxEval v2 | 61.4% | #110 of 129 | Official leaderboard | |
| CorpFin v2 | 46% | #102 of 119 | Official leaderboard | |
| Vals Finance Agent v1 | 4.29% | #43 of 47 | Official leaderboard | |
| Vals Finance Agent | 4.23% | #47 of 49 | Official leaderboard | |
| Cohere enterprise human evaluation | 51.7% | #1 of 1 | Lab model card | |
| Enterprise generative evaluation (Cohere) | 94.2% | #1 of 1 | Lab model card |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| XSTest | 1.10% | #1 of 16 | Lab model card | |
| Default safety evaluation (Cohere) | 70.4% | #1 of 1 | Lab model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Conceptual Reasoning Index | 20.7 | #134 of 151 | Official leaderboard | |
| LMCA | 10.3% | #132 of 147 | Official leaderboard | |
| ACCoRD | 37.3 | #137 of 152 | Official leaderboard |
Its lab's numbers put it at 122.9, number 330 of 404
The Self-Reported Index fits one score to the 20 capability results Cohere reported for this model, on the scale of the Epoch Capabilities Index. Labs choose which benchmarks to show and how to run them, so read it as the lab's claim, not an independent measurement. The 90% interval runs from 120.2 to 125.5. How we compute it.
- RULER 95
- DROP 91.1
- IFEval 90.9
- MGSM 90.1
- MBPP+ 86.2
- MMLU 85.5
- HumanEval 85.4
- MATH 80
- INCLUDE 74.3
- MMLU-Pro 69.6
- Berkeley Function Calling Leaderboard (BFCL) 63.8
- Bird-SQL 59.5
- tau-bench 51.7
- GPQA Diamond 50.8
- BigCodeBench 45.4
- LongBench v2 43.4
- LiveCodeBench v5 26.9
- SWE-bench Verified 26.8
- AIME 2024 23.3
- Aider Polyglot 14.7
Its lab's numbers next to 4 leaderboards
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
| Benchmark | Lab says | Board says | Gap |
|---|---|---|---|
| Bird-SQL | 59.5% Bird Dev execution accuracy (Table 14, Table 1) | 65.7% Leaderboard | −6.2 pts |
| Aider Polyglot | 14.7% Aider Polyglot (Table 13) | 12% Leaderboard, aider, whole edits | +2.7 pts |
| Berkeley Function Calling Leaderboard (BFCL) | 63.8% BFCL Overall from the official leaderboard (Table 5). Live AST 80.5, Multi-turn 25.5 | 64.4% Leaderboard, native function calling | −0.6 pts |
| Cohere enterprise human evaluation | 51.7% Head-to-head human evaluation win rate vs GPT-4o (Nov), average of six enterprise task groups (tie 15.3) | 51.7% Leaderboard | Same |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 35 of them aren't in the launch post.
Capabilities
- MMLU 85.5
- MMLU-Pro 69.6
- GPQA Diamond 50.8
- IFEval 90.9
- InFoBench 94.9
- ChatRAG Bench 72.9
- StrategyQA 76.7
- Bamboogle 76
- DROP 91.1
- HotpotQA 92.1
- Berkeley Function Calling Leaderboard (BFCL) 63.8
- tau-bench 51.7
- NTREX-128 68.8
- FLORES-200 81.2
- m-ArenaHard
- m-tau-bench (Cohere)
- Language Confusion Benchmark 93
- AL-QASIDA
- MBPP+ 86.2
- LiveCodeBench v5 26.9
- BigCodeBench 45.4
- LBPP (Less Basic Python Problems) 51.5
- HumanEvalPack 76.2
- HumanEval 85.4
- HumanEval-COBOL (Cohere) 25.3
- RepoQA 92.6
- SWE-bench Verified 26.8
- Aider Polyglot 14.7
- Spider 80.2
- Bird-SQL 59.5
- Multi-dialect SQL evaluation (Cohere) 55.3
- MATH 80
- AIME 2024 23.3
- Enterprise generative evaluation (Cohere) 94.2
- Enterprise RAG evaluation (Cohere) 4.73
- RULER 95
- LongBench v2 43.4
- Cohere enterprise human evaluation
- MGSM 90.1
- INCLUDE 74.3