Benchmark Almanac

LiveSQLBench

AntQL 1.1 (Claude Opus 5.5) 57% tops its leaderboard MiniMax-M3 40.2% leads the models we track
In use

Contamination-resistant text-to-SQL on PostgreSQL with hierarchical knowledge bases, including CRUD tasks. Scored by soft execution match and test cases.

Gold SQL and test cases are withheld from the public dataset and sent by email request. Base-Full v1 (600 tasks) came out in 2025-09. LiveSQLBench-Large-v1 (March 2026) adds 480 tasks on databases with about 54 tables each. Version 1.5 is announced.

Made by
BIRD team (HKU)
Released
May 2025
Kind
Data science and SQL
Level
University
Scored
Checked automatically
Answers
Code
Tasks
Public, plus a private test set
Strongest model tested
GPT-5.5 #20 on the capability index
Last in a launch post
June 2026
Last model added
June 2026
Scores refreshed
November 12th, 2026
43
Entries on its own leaderboard, 36 models we track
1
Launch post lists it, from 1 lab

AntQL 1.1 (Claude Opus 5.5) tops the leaderboard with 57%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is MiniMax-M3, at 40.2%.

The newest model with a score here came out in June 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

43 entries. Scroll sideways for the rest.

  1. 57.0 AntQL 1.1 (Claude Opus 5.5)
  2. 50.3 AskRUI
  3. 48.0 DIA
  4. 42.8 ShanHe 2.0
  5. 41.3 Jitto Build (Claude Opus 5.5, high)
  6. 40.2 MiniMax-M3
  7. 38.0 Claude Opus 4.6
  8. 37.5 OpenCode-Grok4.7
  9. 36.5 Gemini 3.1 Pro
  10. 35.2 Claude Sonnet 4.5
  11. 33.5 GPT-5.5
  12. 33.2 MiniMax-M2.7
  13. 32.2 Kimi K2.5
  14. 31.0 GPT-5.4
  15. 30.0 GPT-5.3 Codex
  16. 29.3 Kimi K2.6
  17. 29.0 GLM-5.1
  18. 28.7 MiniMax-M2.1
  19. 28.7 Gemini 2.5 Pro
  20. 26.8 Qwen3-Coder-480B-A35B
  21. 26.5 Qwen 3.6 Max (Preview)
  22. 25.5 GPT-5
  23. 25.2 o1
  24. 24.5 DeepSeek-R1
  25. 24.2 o4-mini
  26. 23.8 Gemini 2.5 Flash
  27. 23.7 o3-mini
  28. 23.7 o3
  29. 22.2 Qwen3-235B-A22B
  30. 21.7 Claude 3.7 Sonnet
  31. 20.3 GLM-4.7
  32. 20.0 Claude Sonnet 4
  33. 19.8 DeepSeek-V3
  34. 18.9 Llama 4 Scout
  35. 18.2 QwQ-32B
  36. 15.5 GPT-4o
  37. 13.2 Llama 4 Maverick
  38. 12.5 Qwen2.5-Coder-32B
  39. 12.2 Llama 3.3 70B Instruct
  40. 8.67 Codestral 22B
  41. 6.33 Qwen2.5-Coder (7B)
  42. 2.83 Mistral-7B-Instruct-v0.1
  43. 2.33 Mixtral 8x7B
  1. 1 AntQL 1.1 (Claude Opus 5.5) 57%
  2. 2 AskRUI 50.3%
  3. 3 DIA 48%
  4. 4 ShanHe 2.0 42.8%
  5. 5 Jitto Build (Claude Opus 5.5, high) 41.3%
  6. 6 MiniMax-M3 40.2%
  7. 7 Claude Opus 4.6 38%
  8. 8 OpenCode-Grok4.7 37.5%
  9. 9 Gemini 3.1 Pro 36.5%
  10. 10 Claude Sonnet 4.5 35.2%
  11. 11 GPT-5.5 low 33.5%
  12. 12 MiniMax-M2.7 33.2%
  13. 13 Kimi K2.5 32.2%
  14. 14 GPT-5.4 31%
  15. 15 GPT-5.3 Codex 30%
  16. 16 Kimi K2.6 29.3%
  17. 17 GLM-5.1 29%
  18. 18 MiniMax-M2.1 28.7%
  19. 19 Gemini 2.5 Pro Jun 2025 28.7%
  20. 20 Qwen3-Coder-480B-A35B 26.8%
  21. 21 Qwen 3.6 Max (Preview) 26.5%
  22. 22 GPT-5 25.5%
  23. 23 o1 25.2%
  24. 24 DeepSeek-R1 May 2025 24.5%
  25. 25 o4-mini 24.2%
  26. 26 Gemini 2.5 Flash Sep 2025 23.8%
  27. 27 o3-mini 23.7%
  28. 28 o3 23.7%
  29. 29 Qwen3-235B-A22B 22.2%
  30. 30 Claude 3.7 Sonnet thinking 21.7%
  31. 31 GLM-4.7 20.3%
  32. 32 Claude Sonnet 4 20%
  33. 33 DeepSeek-V3 Mar 2025 19.8%
  34. 34 Llama 4 Scout 18.9%
  35. 35 QwQ-32B 18.2%
  36. 36 GPT-4o Mar 2025 15.5%
  37. 37 Llama 4 Maverick 13.2%
  38. 38 Qwen2.5-Coder-32B 12.5%
  39. 39 Llama 3.3 70B Instruct 12.2%
  40. 40 Codestral 22B 8.67%
  41. 41 Qwen2.5-Coder (7B) 6.33%
  42. 42 Mistral-7B-Instruct-v0.1 2.83%
  43. 43 Mixtral 8x7B 2.33%

Source: Official LiveSQLBench leaderboard

#EntryScoreSettingDateSource
1 AntQL 1.1 (Claude Opus 5.5) 57% Official leaderboard
2 AskRUI 50.3% Official leaderboard
3 DIA 48% Data Intelligence Agents Official leaderboard
4 ShanHe 2.0 42.8% Official leaderboard
5 Jitto Build (Claude Opus 5.5, high) 41.3% Official leaderboard
6 MiniMax-M3 40.2% Claude Code Jun 2026 Official leaderboard
7 Claude Opus 4.6 38% OpenHands CLI Feb 2026 Official leaderboard
8 OpenCode-Grok4.7 37.5% Official leaderboard
9 Gemini 3.1 Pro 36.5% Feb 2026 Official leaderboard
10 Claude Sonnet 4.5 35.2% OpenHands CLI Sep 2025 Official leaderboard
11 GPT-5.5 33.5% low Apr 2026 Official leaderboard
12 MiniMax-M2.7 33.2% Claude Code Mar 2026 Official leaderboard
13 Kimi K2.5 32.2% OpenHands CLI Jan 2026 Official leaderboard
14 GPT-5.4 31% Mar 2026 Official leaderboard
15 GPT-5.3 Codex 30% Feb 2026 Official leaderboard
16 Kimi K2.6 29.3% Apr 2026 Official leaderboard
17 GLM-5.1 29% Apr 2026 Official leaderboard
18 MiniMax-M2.1 28.7% OpenHands CLI Dec 2025 Official leaderboard
19 Gemini 2.5 Pro Jun 2025 28.7% Jun 2025 Official leaderboard
20 Qwen3-Coder-480B-A35B 26.8% OpenHands CLI Jul 2025 Official leaderboard
21 Qwen 3.6 Max (Preview) 26.5% Apr 2026 Official leaderboard
22 GPT-5 25.5% Aug 2025 Official leaderboard
23 o1 25.2% Dec 2024 Official leaderboard
24 DeepSeek-R1 May 2025 24.5% May 2025 Official leaderboard
25 o4-mini 24.2% Apr 2025 Official leaderboard
26 Gemini 2.5 Flash Sep 2025 23.8% Sep 2025 Official leaderboard
27 o3-mini 23.7% Jan 2025 Official leaderboard
28 o3 23.7% Apr 2025 Official leaderboard
29 Qwen3-235B-A22B 22.2% Apr 2025 Official leaderboard
30 Claude 3.7 Sonnet 21.7% thinking Feb 2025 Official leaderboard
31 GLM-4.7 20.3% OpenHands CLI Dec 2025 Official leaderboard
32 Claude Sonnet 4 20% May 2025 Official leaderboard
33 DeepSeek-V3 Mar 2025 19.8% Mar 2025 Official leaderboard
34 Llama 4 Scout 18.9% Apr 2025 Official leaderboard
35 QwQ-32B 18.2% Mar 2025 Official leaderboard
36 GPT-4o Mar 2025 15.5% Mar 2025 Official leaderboard
37 Llama 4 Maverick 13.2% Apr 2025 Official leaderboard
38 Qwen2.5-Coder-32B 12.5% Sep 2024 Official leaderboard
39 Llama 3.3 70B Instruct 12.2% Dec 2024 Official leaderboard
40 Codestral 22B 8.67% Official leaderboard
41 Qwen2.5-Coder (7B) 6.33% Sep 2024 Official leaderboard
42 Mistral-7B-Instruct-v0.1 2.83% Sep 2023 Official leaderboard
43 Mixtral 8x7B 2.33% Dec 2023 Official leaderboard

Labs report scores for 1 model in their cards. We plot every source in the charts below.

Of runs with a cost, the best scores 36.5% for $0.05 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Qwen Meta DeepSeek OpenAI Google DeepMind Moonshot AI Cost frontier One model at each effort level
0% 10% 20% 30% 40% $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Gemini 3.1 Pro Kimi K2.6 GPT-5 DeepSeek-R1 Gemini 2.5 Flash Qwen3-235B-A22B Qwen3-Coder-480B-A35B 0% 10% 20% 30% 40% $0.0001 $0.001 $0.01 $0.10 $1.00 Cost per task, log scale Gemini 3.1 Pro Kimi K2.6

Source: Official LiveSQLBench leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
MiniMax-M3 Claude Code 40.2%
Claude Opus 4.6 OpenHands CLI 38%
Gemini 3.1 Pro 36.5% $0.05
Claude Opus 4.6 35.5% $0.10
Claude Sonnet 4.5 OpenHands CLI 35.2%
GPT-5.5 low 33.5% $0.09
GPT-5.5 xhigh 33.3% $0.22
MiniMax-M2.7 Claude Code 33.2%
Kimi K2.5 OpenHands CLI 32.2%
GPT-5.4 31%
GPT-5.3 Codex 30%
Kimi K2.6 29.3% $0.04
GLM-5.1 29%
MiniMax-M2.1 OpenHands CLI 28.7%
Gemini 2.5 Pro Jun 2025 28.7% $0.05
Qwen3-Coder-480B-A35B OpenHands CLI 26.8%
Qwen 3.6 Max (Preview) 26.5%
GPT-5 25.5% $0.02
o1 25.2% $0.23
DeepSeek-R1 May 2025 24.5% $0.01
o4-mini 24.2% $0.02
Gemini 2.5 Flash Sep 2025 23.8% $0.005
Claude Sonnet 4.5 23.8% $0.06
o3-mini 23.7% $0.02
o3 23.7% $0.16
Kimi K2.5 23% $0.01
Qwen3-235B-A22B 22.2% $0.004
Claude 3.7 Sonnet thinking 21.7% $0.12
GLM-4.7 OpenHands CLI 20.3%
Claude Sonnet 4 20% $0.06
DeepSeek-V3 Mar 2025 19.8% $0.004
Claude 3.7 Sonnet 19.7% $0.06
Qwen3-Coder-480B-A35B 19.2% $0.004
Llama 4 Scout 18.9%
MiniMax-M2.1 18.8% $0.006
QwQ-32B 18.2% $0.001
GLM-4.7 18.2% $0.006
GPT-4o Mar 2025 15.5% $0.04
Llama 4 Maverick 13.2% $0.003
Qwen2.5-Coder-32B 12.5% $0.0008
Llama 3.3 70B Instruct 12.2% $0.0006
Qwen2.5-Coder (7B) 6.33% $0.002
Mistral-7B-Instruct-v0.1 2.83% $0.003
Mixtral 8x7B 2.33% $0.001

5 models have held number one on the official leaderboard since its release in May 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Google DeepMind Anthropic MiniMax
Own leaderboard Not tested since Jun 2026
MiniMax-M3 40.2%

Source: Official LiveSQLBench leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) MiniMax-M3 Claude Sonnet 4.5 Gemini 2.5 Pro o1 Qwen2.5-Coder-32B Mistral-7B-Instruct-v0.1 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) MiniMax-M3 Gemini 2.5 Pro

Sources: Official LiveSQLBench leaderboard and labs’ model cards

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

MiniMax
DateLaunch postScoreAs the post writes it
Jun 1, 2026 MiniMax-M3 40.2 LiveSQLBench

One lab claim next to the leaderboard

Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. 1 of them match, which usually means one side copied the other. Every benchmark.

Model Lab says Board says Gap
MiniMax-M3 40.2% LiveSQLBench 40.2% Leaderboard, Claude Code Same

More in Coding

All 282 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Fable 5.2 96.6%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471