Benchmark Almanac

CoQA

RoBERTa + AT + KD (ensemble) 90.7% tops its leaderboard
Not in model cards

Models answer a series of linked conversational questions about a passage, where later questions depend on earlier turns. Scored by word-level F1.

Reported in early LLM papers such as GPT-3.

Released
August 2018
Kind
Reading comprehension
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
Public, plus a private test set

In the authors’ words

Humans gather information by engaging in conversations involving a series of interconnected questions and answers. For machines to assist in information gathering, it is therefore essential to enable them to answer conversational questions. We introduce CoQA, a novel dataset for building Conversational Question Answering systems. Our dataset contains 127k questions with answers, obtained from 8k conversations about text passages from seven diverse domains. The questions are conversational, and the answers are free-form text with their corresponding evidence highlighted in the passage. We analyze CoQA in depth and show that conversational questions have challenging phenomena not present in existing reading comprehension datasets, e.g., coreference and pragmatic reasoning. We evaluate strong conversational and reading comprehension models on CoQA. The best system obtains an F1 score of 65.4%, which is 23.4 points behind human performance (88.8%), indicating there is ample room for improvement. We launch CoQA as a challenge to the community at http://stanfordnlp.github.io/coqa/

Abstract of CoQA: A Conversational Question Answering Challenge (arXiv:1808.07042, CC0)
42
Entries on its own leaderboard, 0 models we track

RoBERTa + AT + KD (ensemble) tops the leaderboard with 90.7%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

42 entries. Scroll sideways for the rest.

  1. 90.7 RoBERTa + AT + KD (ensemble)
  2. 90.7 TR-MT (ensemble)
  3. 90.4 RoBERTa + AT + KD (single model)
  4. 89.4 Google SQuAD 2.0 + MMFT (ensemble)
  5. 89.3 TR-MT (single model)
  6. 89.0 XLNet + Augmentation (single model)
  7. 87.8 ConvBERT (ensemble)
  8. 87.8 Google SQuAD 2.0 + MMFT (single model)
  9. 86.8 BERT + MMFT + ADA (ensemble)
  10. 86.8 ConvBERT (single model)
  11. 85.0 BERT + MMFT + ADA (single model)
  12. 84.6 XLNet + MMFT + ADA (single model)
  13. 83.0 BERT + AttentionFusionNet (single model)
  14. 82.8 BERT + Answer Verification (single model)
  15. 81.5 BERT with History Augmented Query (single model)
  16. 81.4 BERT Large Finetuned Baseline (single model)
  17. 81.1 BERT Large Augmented (single model)
  18. 80.2 D-AoA + BERT (single model)
  19. 80.0 BERT Augmented + AoA (single model)
  20. 79.8 CNet (single model)
  21. 79.3 SDNet (ensemble)
  22. 79.1 CQANet (single model)
  23. 78.9 CANet (single model)
  24. 78.7 BERT Finetuned Baseline
  25. 78.7 BERT w/ 2-context (single model)
  26. 78.2 Bert-MultiChannelFlow (single model)
  27. 78.1 BERT-base finetune (single model)
  28. 77.7 Bert-FlowDelta (single model)
  29. 77.3 GraphFlow (single model)
  30. 76.6 SDNet (single model)
  31. 75.8 Flow Framework (single model)
  32. 75.0 FlowQA (single model)
  33. 74.4 HisFurC + BERT
  34. 73.3 RNet + PGNet + BERT (single model)
  35. 72.7 XyzNet (single model)
  36. 69.7 DrQA + marker features (single model)
  37. 69.5 BiDAF++ (single model)
  38. 66.5 Bert Base Augmented (single model)
  39. 66.4 RNet_DotAtt + seq2seq with copy attention (single model)
  40. 66.3 Simplified BiDAF++ (single model)
  41. 65.1 DrQA + seq2seq with copy attention (single model)
  42. 52.6 Vanilla DrQA (single model)
  1. 1 RoBERTa + AT + KD (ensemble) 90.7%
  2. 2 TR-MT (ensemble) 90.7%
  3. 3 RoBERTa + AT + KD (single model) 90.4%
  4. 4 Google SQuAD 2.0 + MMFT (ensemble) 89.4%
  5. 5 TR-MT (single model) 89.3%
  6. 6 XLNet + Augmentation (single model) 89%
  7. 7 ConvBERT (ensemble) 87.8%
  8. 8 Google SQuAD 2.0 + MMFT (single model) 87.8%
  9. 9 BERT + MMFT + ADA (ensemble) 86.8%
  10. 10 ConvBERT (single model) 86.8%
  11. 11 BERT + MMFT + ADA (single model) 85%
  12. 12 XLNet + MMFT + ADA (single model) 84.6%
  13. 13 BERT + AttentionFusionNet (single model) 83%
  14. 14 BERT + Answer Verification (single model) 82.8%
  15. 15 BERT with History Augmented Query (single model) 81.5%
  16. 16 BERT Large Finetuned Baseline (single model) 81.4%
  17. 17 BERT Large Augmented (single model) 81.1%
  18. 18 D-AoA + BERT (single model) 80.2%
  19. 19 BERT Augmented + AoA (single model) 80%
  20. 20 CNet (single model) 79.8%
  21. 21 SDNet (ensemble) 79.3%
  22. 22 CQANet (single model) 79.1%
  23. 23 CANet (single model) 78.9%
  24. 24 BERT Finetuned Baseline 78.7%
  25. 25 BERT w/ 2-context (single model) 78.7%
  26. 26 Bert-MultiChannelFlow (single model) 78.2%
  27. 27 BERT-base finetune (single model) 78.1%
  28. 28 Bert-FlowDelta (single model) 77.7%
  29. 29 GraphFlow (single model) 77.3%
  30. 30 SDNet (single model) 76.6%
  31. 31 Flow Framework (single model) 75.8%
  32. 32 FlowQA (single model) 75%
  33. 33 HisFurC + BERT 74.4%
  34. 34 RNet + PGNet + BERT (single model) 73.3%
  35. 35 XyzNet (single model) 72.7%
  36. 36 DrQA + marker features (single model) 69.7%
  37. 37 BiDAF++ (single model) 69.5%
  38. 38 Bert Base Augmented (single model) 66.5%
  39. 39 RNet_DotAtt + seq2seq with copy attention (single model) 66.4%
  40. 40 Simplified BiDAF++ (single model) 66.3%
  41. 41 DrQA + seq2seq with copy attention (single model) 65.1%
  42. 42 Vanilla DrQA (single model) 52.6%

Source: Official CoQA leaderboard

#EntryScoreSettingDateSource
1 RoBERTa + AT + KD (ensemble) 90.7% Official leaderboard
2 TR-MT (ensemble) 90.7% Official leaderboard
3 RoBERTa + AT + KD (single model) 90.4% Official leaderboard
4 Google SQuAD 2.0 + MMFT (ensemble) 89.4% Official leaderboard
5 TR-MT (single model) 89.3% Official leaderboard
6 XLNet + Augmentation (single model) 89% Official leaderboard
7 ConvBERT (ensemble) 87.8% Official leaderboard
8 Google SQuAD 2.0 + MMFT (single model) 87.8% Official leaderboard
9 BERT + MMFT + ADA (ensemble) 86.8% Official leaderboard
10 ConvBERT (single model) 86.8% Official leaderboard
11 BERT + MMFT + ADA (single model) 85% Official leaderboard
12 XLNet + MMFT + ADA (single model) 84.6% Official leaderboard
13 BERT + AttentionFusionNet (single model) 83% Official leaderboard
14 BERT + Answer Verification (single model) 82.8% Official leaderboard
15 BERT with History Augmented Query (single model) 81.5% Official leaderboard
16 BERT Large Finetuned Baseline (single model) 81.4% Official leaderboard
17 BERT Large Augmented (single model) 81.1% Official leaderboard
18 D-AoA + BERT (single model) 80.2% Official leaderboard
19 BERT Augmented + AoA (single model) 80% Official leaderboard
20 CNet (single model) 79.8% Official leaderboard
21 SDNet (ensemble) 79.3% Official leaderboard
22 CQANet (single model) 79.1% Official leaderboard
23 CANet (single model) 78.9% Official leaderboard
24 BERT Finetuned Baseline 78.7% Official leaderboard
25 BERT w/ 2-context (single model) 78.7% Official leaderboard
26 Bert-MultiChannelFlow (single model) 78.2% Official leaderboard
27 BERT-base finetune (single model) 78.1% Official leaderboard
28 Bert-FlowDelta (single model) 77.7% Official leaderboard
29 GraphFlow (single model) 77.3% Official leaderboard
30 SDNet (single model) 76.6% Official leaderboard
31 Flow Framework (single model) 75.8% Official leaderboard
32 FlowQA (single model) 75% Official leaderboard
33 HisFurC + BERT 74.4% Official leaderboard
34 RNet + PGNet + BERT (single model) 73.3% Official leaderboard
35 XyzNet (single model) 72.7% Official leaderboard
36 DrQA + marker features (single model) 69.7% Official leaderboard
37 BiDAF++ (single model) 69.5% Official leaderboard
38 Bert Base Augmented (single model) 66.5% Official leaderboard
39 RNet_DotAtt + seq2seq with copy attention (single model) 66.4% Official leaderboard
40 Simplified BiDAF++ (single model) 66.3% Official leaderboard
41 DrQA + seq2seq with copy attention (single model) 65.1% Official leaderboard
42 Vanilla DrQA (single model) 52.6% Official leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

More in Knowledge

All 121 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • KMMLU-Pro In use Models answer multiple-choice questions taken from Korean National Professional Licensure exams. It tests professional knowledge used in Korean industry. Solar Pro 4 79.2%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Seed2.0 Pro 73.6%