Benchmark Almanac

Valyrian Games Coding Challenge

GPT-5 mini 30 leads since September 2025
Not in model cards

Models create coding problems with integer answers and repeatedly solve their own problems to qualify. Tournament entrants solve one another’s problems and receive TrueSkill ratings.

The organizer ranks 18 qualifying models after 50 tournaments. The qualification round tested 62 models.

Made by
Valyrian Tech
Released
September 2025
Kind
Competitive programming
Level
Competition
Scored
Ranked by matches
Answers
Agent task
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Code, Text
Interaction
Multi-agent
Environment
Code sandbox
Refresh
Live
Run by
A host
In views
Agents, Kept fresh
Strongest model tested
GPT-5 mini #92 on the capability index
Last model added
August 2025
Scores refreshed
November 20th, 2026
18
Entries on its own leaderboard, 17 models we track

GPT-5 mini leads with 30 among models released by August 2025 GPT-5 mini leads with 30 among models released by August 2025

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. This view shows only models we track, each at its best.

The newest model with a score here came out in August 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

Bars start at 10.

  1. 1 GPT-5 mini 30
  2. 2 o4-mini 29
  3. 3 GPT-4.1 29
  4. 4 o1 29
  5. 5 DeepSeek:deepseek-chat 29
  6. 6 Claude Opus 4.1 28
  7. 7 Claude Sonnet 4 28
  8. 8 Claude Opus 4 28
  9. 9 gpt-oss-120b 28
  10. 10 Qwen3-235B-A22B-Instruct Jul 2025 28
  11. 11 Qwen3-Coder-480B-A35B 28
  12. 12 Codestral 27
  13. 13 GPT-4o Aug 2024 26
  14. 14 o3-mini 23
  15. 15 Pixtral Large 23
  16. 16 Qwen3-235B-A22B-Thinking Jul 2025 17
  17. 17 Magistral Medium 1.0 13
  18. 18 Devstral Medium 13

Source: Official Valyrian Games Coding Challenge leaderboard

#EntryScoreSettingDateSource
1 GPT-5 mini 30 Aug 2025 Official leaderboard
2 o4-mini 29 Apr 2025 Official leaderboard
3 GPT-4.1 29 Apr 2025 Official leaderboard
4 o1 29 Dec 2024 Official leaderboard
5 DeepSeek:deepseek-chat 29 Official leaderboard
6 Claude Opus 4.1 28 Aug 2025 Official leaderboard
7 Claude Sonnet 4 28 May 2025 Official leaderboard
8 Claude Opus 4 28 May 2025 Official leaderboard
9 gpt-oss-120b 28 Aug 2025 Official leaderboard
10 Qwen3-235B-A22B-Instruct Jul 2025 28 Jul 2025 Official leaderboard
11 Qwen3-Coder-480B-A35B 28 Jul 2025 Official leaderboard
12 Codestral 27 Jan 2025 Official leaderboard
13 GPT-4o Aug 2024 26 Aug 2024 Official leaderboard
14 o3-mini 23 Jan 2025 Official leaderboard
15 Pixtral Large 23 Nov 2024 Official leaderboard
16 Qwen3-235B-A22B-Thinking Jul 2025 17 Jul 2025 Official leaderboard
17 Magistral Medium 1.0 13 Jun 2025 Official leaderboard
18 Devstral Medium 13 Jul 2025 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-5 mini 30 Aug 2025 Official leaderboard
2 o4-mini 29 Apr 2025 Official leaderboard
3 GPT-4.1 29 Apr 2025 Official leaderboard
4 o1 29 Dec 2024 Official leaderboard
5 Claude Opus 4.1 28 Aug 2025 Official leaderboard
6 Claude Sonnet 4 28 May 2025 Official leaderboard
7 Claude Opus 4 28 May 2025 Official leaderboard
8 gpt-oss-120b 28 Aug 2025 Official leaderboard
9 Qwen3-235B-A22B-Instruct Jul 2025 28 Jul 2025 Official leaderboard
10 Qwen3-Coder-480B-A35B 28 Jul 2025 Official leaderboard
11 Codestral 27 Jan 2025 Official leaderboard
12 GPT-4o Aug 2024 26 Aug 2024 Official leaderboard
13 o3-mini 23 Jan 2025 Official leaderboard
14 Pixtral Large 23 Nov 2024 Official leaderboard
15 Qwen3-235B-A22B-Thinking Jul 2025 17 Jul 2025 Official leaderboard
16 Magistral Medium 1.0 13 Jun 2025 Official leaderboard
17 Devstral Medium 13 Jul 2025 Official leaderboard

The best score, 30, costs $0.06 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Mistral AI Qwen OpenAI Cost frontier One model at each effort level
0 10 20 30 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5 mini gpt-oss-120b Devstral Medium 0 10 20 30 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5 mini gpt-oss-120b

Source: Official Valyrian Games Coding Challenge leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5 mini 30 $0.06
o4-mini 29 $0.13
GPT-4.1 29 $0.13
o1 29 $2.24
Claude Opus 4.1 28 $2.08
Claude Sonnet 4 28 $0.98
Claude Opus 4 28 $2.41
gpt-oss-120b 28 $0.02
Qwen3-235B-A22B-Instruct Jul 2025 28 $0.01
Qwen3-Coder-480B-A35B 28 $0.11
Codestral 27 $0.02
GPT-4o Aug 2024 26 $0.20
o3-mini 23 $0.08
Pixtral Large 23 $0.22
Qwen3-235B-A22B-Thinking Jul 2025 17 $0.13
Magistral Medium 1.0 13 $0.69
Devstral Medium 13 $0.009

The best score, 30, costs $0.06 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Mistral AI Qwen OpenAI Cost frontier One model at each effort level
0 10 20 30 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5 mini gpt-oss-120b Devstral Medium 0 10 20 30 $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5 mini gpt-oss-120b

Source: Official Valyrian Games Coding Challenge leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5 mini 30 $0.06
o4-mini 29 $0.13
GPT-4.1 29 $0.13
o1 29 $2.24
Claude Opus 4.1 28 $2.08
Claude Sonnet 4 28 $0.98
Claude Opus 4 28 $2.41
gpt-oss-120b 28 $0.02
Qwen3-235B-A22B-Instruct Jul 2025 28 $0.01
Qwen3-Coder-480B-A35B 28 $0.11
Codestral 27 $0.02
GPT-4o Aug 2024 26 $0.20
o3-mini 23 $0.08
Pixtral Large 23 $0.22
Qwen3-235B-A22B-Thinking Jul 2025 17 $0.13
Magistral Medium 1.0 13 $0.69
Devstral Medium 13 $0.009

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Sep 2025

Source: Official Valyrian Games Coding Challenge leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
10 15 20 25 30 2025 10 15 20 25 30 2025

Source: Official Valyrian Games Coding Challenge leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Coding

All 523 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471

Data audit

8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 15 months ago, in August 2025, for GPT-5 mini. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (18 entries).
  • Pass Models scored 17 models we track, plus 1 other entry on its leaderboard.
  • Warning Newest score The newest came 15 months ago, in August 2025, for GPT-5 mini.
  • Note Frontier models None of the top 10 has a score. The strongest tested is GPT-5 mini (#92).
  • Note Headroom Scored as a rating, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on valyriantech.github.io: 17 rows for models we track and 1 other entry at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check