Spider

Qwen3-Coder-Next 83.7% leads since February 2026

T5-3B 78% leads since October 2019

Qwen3-Coder-Next 83.7% leads since February 2026

Dropped

Labs’ own numbers All independent runs Include claimed scores

Models turn natural language questions into SQL over 200 multi-table databases from 138 domains. Test databases are unseen in training, and answers are scored by exact match or execution.

10,181 questions and 5,693 unique SQL queries annotated by 11 Yale students. The Spider site announced Spider 2.0 as its successor in 2024.

Made by: Yale University (LILY lab, Tao Yu et al.)
Released: September 2018
Kind: Data science and SQL
Level: University
Scored: Checked automatically
Answers: Code
Tasks: Public, plus a private test set

Last in a launch post: January 2025, dropped for a newer version
Last model added: February 2026
Scores refreshed: November 12th, 2026

In the authors’ words

We present Spider, a large-scale, complex and cross-domain semantic parsing and text-to-SQL dataset annotated by 11 college students. It consists of 10,181 questions and 5,693 unique complex SQL queries on 200 databases with multiple tables, covering 138 different domains. We define a new complex and cross-domain semantic parsing and text-to-SQL task where different complex SQL queries and databases appear in train and test sets. In this way, the task requires the model to generalize well to both new SQL queries and new database schemas. Spider is distinct from most of the previous semantic parsing tasks because they all use a single database and the exact same programs in the train set and the test set. We experiment with various state-of-the-art models and the best model achieves only 12.4% exact matching accuracy on a database split setting. This shows that Spider presents a strong challenge for future research. Our dataset and task are publicly available at https://yale-lily.github.io/spider

Abstract of Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task (arXiv:1809.08887, CC0)

Models scored in lab cards

Launch post lists it, from 1 lab

System cards report it, from 3 labs

Qwen3-Coder-Next leads with 83.7% among models released by February 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

T5-3B leads with 78% across all independent runs

Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Each bar and row names its source.

Qwen3-Coder-Next leads with 83.7%, claims included

Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model.

Bars Table

The newest model with a score here came out in February 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

1 Qwen3-Coder-Next 83.7%
2 Cohere Command A 80.2%
3 Codestral 66.5%
4 Phi-4 Mini 42.2%

Source: Labs’ model cards

1 T5-3B 78%
2 T5-Base 71.1%

Source: Official Spider leaderboard

1 Qwen3-Coder-Next 83.7%
2 Cohere Command A 80.2%
3 T5-3B 78%
4 T5-Base 71.1%
5 Codestral 66.5%
6 Phi-4 Mini 42.2%

Sources: Labs’ model cards and official Spider leaderboard

#	Model	Score	Date	Source
1	Qwen3-Coder-Next	83.7%	Feb 2026	Lab model card
2	Cohere Command A	80.2%	Apr 2025	Lab model card
3	Codestral	66.5%	Jan 2025	Lab model card
4	Phi-4 Mini	42.2%	Mar 2025	Lab model card

#	Model	Score	Setting	Date	Source
1	T5-3B	78%	T5-3B+NatSQL+Token Preprocessing	Oct 2019	Official leaderboard
2	T5-Base	71.1%	T5-Base+NatSQL+Token Preprocessing	Aug 2021	Official leaderboard

#	Model	Score	Setting	Date	Source
1	Qwen3-Coder-Next	83.7%		Feb 2026	Lab model card
2	Cohere Command A	80.2%		Apr 2025	Lab model card
3	T5-3B	78%	T5-3B+NatSQL+Token Preprocessing	Oct 2019	Official leaderboard
4	T5-Base	71.1%	T5-Base+NatSQL+Token Preprocessing	Aug 2021	Official leaderboard
5	Codestral	66.5%		Jan 2025	Lab model card
6	Phi-4 Mini	42.2%		Mar 2025	Lab model card

The benchmark's own leaderboard has 29 entries, 2 of them models we track. Switch the scores above to rank them together.

3 models have held number one in labs’ own numbers since 2023 Number one over time 3 models have held number one with claims included since 2023

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Google DeepMind Mistral AI Cohere Qwen

Own leaderboard Not tested since Aug 2021

T5-3B 78%

Lab claims Not tested since Feb 2026

Qwen3-Coder-Next 83.7%

Sources: Official Spider leaderboard and labs’ model cards

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far

Sources: Official Spider leaderboard and labs’ model cards

Labs moved on to Spider 2.0

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Mistral AI

Date	Launch post	Score	As the post writes it
Jan 13, 2025	Codestral	66.5	Spider (SQL)

3 system cards report it, from 3 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
Feb 28, 2026	Qwen3-Coder-Next technical report	83.7	Spider (Table 7)	Capabilities
Apr 1, 2025	Cohere Command A technical report	80.2	Spider Test execution accuracy (Table 14). Dev 79.5	Capabilities
Mar 3, 2025	Phi-4 Mini technical report	42.2	Spider (4-Shot) (Table 8)	Capabilities

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Mistral AI Cohere Qwen OpenAI Anthropic Google DeepMind

Spider 2.0 Passed 85% in Sep 2025, by Claude Sonnet 4.5

Gemini 3 Pro 94.2%

Spider Not tested since Feb 2026