Command R (03-2024)
Command R (03-2024) places in the top three on 2 benchmarks.
24
Benchmarks with a score
0
Ranked first, out of five or more models
2
In the top three
Cohere Mar 2024 Command R (03-2024)
Open weights Mid-size
No Epoch Capabilities Index score yet
- Open Arabic LLM Leaderboard v1 54.4% #5 /55
- ThaiExam 37.7% #20 /24
- PHYSICS 7.00% #17 /20
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open Arabic LLM Leaderboard v1 | 54.4% | #5 of 55 | Official leaderboard | |
| MuSR | 45.2% | #15 of 98 | Official leaderboard | |
| BABILong | 59.1% | #3 of 6 | Official leaderboard | |
| RULER | 88.3% | #9 of 15 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.90% | #94 of 148 | Official leaderboard | |
| WildBench | 35.1 | #30 of 47 | Official leaderboard | |
| MixEval | 77% | #24 of 34 | Official leaderboard | |
| MMLU-Pro | 37.9% | #169 of 212 | Official leaderboard | |
| LMArena (Elo) | 1,227 | #272 of 335 | Official leaderboard | |
| ThaiExam | 37.7% | #20 of 24 | Official leaderboard | |
| EQ-Bench v2 | 56.0 | #73 of 85 | Official leaderboard | |
| SnorkelSpatial | 11.8% | #27 of 27 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ZebraLogic | 9.90% | #37 of 46 | Official leaderboard | |
| PHYSICS | 7.00% | #17 of 20 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 55.5% | #56 of 71 | Official leaderboard | |
| LegalBench | 33% | #138 of 138 | Official leaderboard | |
| SnorkelUnderwrite | 15.3% | #27 of 27 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AgentDojo | 3.34% | #2 of 20 | Official leaderboard | |
| SimpleSafetyTests | 94.3% | #65 of 75 | Official leaderboard | |
| Anthropic Red Team | 93.7% | #70 of 75 | Official leaderboard | |
| HELM Safety | 80.9 | #72 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 31.8% | #80 of 82 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Open LLM Leaderboard v2 | 25.9 | #40 of 99 | Official leaderboard | |
| HELM Lite | 29.9 | #59 of 71 | Official leaderboard |