Palmyra X4
Palmyra X4.
12
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Writer Oct 2024
Palmyra X4
Closed weights Flagship
No Epoch Capabilities Index score yet
- GPQA Main 39.5% #45 /60
- AIR-Bench 2024 44.2% #71 /82
- Anthropic Red Team 98.6% #47 /75
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| IFEval | 87.3% | #16 of 60 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GPQA Main | 39.5% | #45 of 60 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Galileo Agent Leaderboard | 0.89 | #12 of 29 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 56.8% | #34 of 67 | Official leaderboard | |
| BFCL V4 | 27.9% | #44 of 65 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 77.5% | #13 of 71 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SimpleSafetyTests | 100% | #7 of 75 | Official leaderboard | |
| HELM Safety | 93.3 | #30 of 75 | Official leaderboard | |
| Anthropic Red Team | 98.6% | #47 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 44.2% | #71 of 82 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 80.8 | #15 of 71 | Official leaderboard | |
| HELM Capabilities | 60.9 | #40 of 60 | Official leaderboard |