Meet new Sonar (Perplexity blog) Feb 11, 2025
- Factuality 85.1
- Readability 85.9
- IFEval 86.8
- MMLU (0-shot) 87.1
sonar
sonar.
7
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Perplexity Jan 2025 sonar
Closed weights
No Epoch Capabilities Index score yet
- IFEval 86.8% #65 /135
- MMLU 87.1% #40 /131
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU | 87.1% | #40 of 131 | Lab model card | |
| IFEval | 86.8% | #65 of 135 | Lab model card | |
| Wer wird Millionär? LLM Benchmark | 3956.0 | #38 of 77 | Official leaderboard | |
| StudyArena Text | 1,228 | #47 of 85 | Official leaderboard | |
| Answer Factuality (Perplexity) | 85.1% | #1 of 1 | Lab model card | |
| Readability (Perplexity) | 85.9% | #1 of 1 | Lab model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MTEB | 42% | #1 of 1 | Official leaderboard |
Its lab's number next to a leaderboard
Where the benchmark's own leaderboard, or Epoch AI's run, also has this model. Gaps mostly come from different harnesses, effort levels and task subsets. Every pair.
| Benchmark | Lab says | Board says | Gap |
|---|---|---|---|
| Answer Factuality (Perplexity) | 85.1% Factuality | 85.1% Leaderboard | Same |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.