Benchmark Almanac

SemEval 2026 Task 4: Narrative Story Similarity and Narrative Representation Learning

COGNAC 78% tops its leaderboard
Not in model cards

Models choose which story is more similar to a reference story. A second track compares story embeddings. Each track scores accuracy on 400 triples.

The organizers ranked 46 teams across two tracks. COGNAC used GPT-4.1 mini and Gemini 2.5 Flash and led both tracks.

Released
July 2025
Kind
Reading comprehension
Level
School
Scored
Checked automatically
Answers
Multiple choice
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Refresh
One fixed set
Run by
Anyone
46
Entries on its own leaderboard, 0 models we track

COGNAC tops the leaderboard with 78%

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

  1. 1 COGNAC Track A 78%
  2. 2 FactUEP Track A 75.8%
  3. 3 AI-Monitors Track A 75%
  4. 4 TeleAI Track A 74.8%
  5. 5 YNU-HPCC Track A 74.3%
  6. 6 UTD-HLTRI Track A 74%
  7. 7 JCT Track A 73.8%
  8. 8 CuriosAI Track A 73.5%
  9. 9 Yam Track A 73%
  10. 10 CascadeMind Track A 72.8%
  11. 11 Team CV Track A 70.8%
  12. 12 hermeneutic_hools Track A 70.5%
  13. 13 hits_team Track B 70.5%
  14. 14 NarSiL Track A 70.3%
  15. 15 NCL&HKU-NarrSim Track A 70.3%
  16. 16 DUTIR Track A 69.8%
  17. 17 CophiWue Track A 69.5%
  18. 18 CITD@UIT Track A 69.3%
  19. 19 Narrative Team Track B 69.3%
  20. 20 ttda704 Track A 69.3%
  21. 21 Narrative Nexus Track A 68.5%
  22. 22 Comhis Track A 68.3%
  23. 23 harapalb Track A 68.3%
  24. 24 SoloSemantics Track A 68%
  25. 25 schmerle Track A 67.8%
  26. 26 L3IRIT Track A 65.8%
  27. 27 MarSan Track B 65.5%
  28. 28 NLP-FSDM Track A 65.5%
  29. 29 Team UBSE Track A 65.3%
  30. 30 MoodMetric Track A 65%
  31. 31 PEU Lab Track A 64.5%
  32. 32 CUNI Track B 64.3%
  33. 33 ChulaNLP Track A 63.5%
  34. 34 blue Track A 62.5%
  35. 35 Team HausaNLP Track A 61.5%
  36. 36 TFB Track B 61.5%
  37. 37 Spinfo Cologne Track A 60.3%
  38. 38 LIAAD INESCTEC Track A 59.8%
  39. 39 Cryptix Track A 59.5%
  40. 40 VerbaNexAI Track B 59.3%
  41. 41 CICL26 Track A 59%
  42. 42 Duluth Track A 58.5%
  43. 43 Mendel292 Track B 58%
  44. 44 IIITH Boys Track A 57.8%
  45. 45 Lacuna Inc. Track A 57.3%
  46. 46 PLlama Track A 55.5%

Source: Official SemEval 2026 Task 4: Narrative Story Similarity and Narrative Representation Learning leaderboard

#EntryScoreSettingSubmitted byDateLinkSource
1 COGNAC 78% Track A COGNAC aclanthology.org Official leaderboard
2 FactUEP 75.8% Track A FactUEP aclanthology.org Official leaderboard
3 AI-Monitors 75% Track A AI-Monitors aclanthology.org Official leaderboard
4 TeleAI 74.8% Track A TeleAI aclanthology.org Official leaderboard
5 YNU-HPCC 74.3% Track A YNU-HPCC aclanthology.org Official leaderboard
6 UTD-HLTRI 74% Track A UTD-HLTRI aclanthology.org Official leaderboard
7 JCT 73.8% Track A JCT aclanthology.org Official leaderboard
8 CuriosAI 73.5% Track A CuriosAI aclanthology.org Official leaderboard
9 Yam 73% Track A Yam aclanthology.org Official leaderboard
10 CascadeMind 72.8% Track A CascadeMind aclanthology.org Official leaderboard
11 Team CV 70.8% Track A Team CV aclanthology.org Official leaderboard
12 hermeneutic_hools 70.5% Track A hermeneutic_hools aclanthology.org Official leaderboard
13 hits_team 70.5% Track B hits_team aclanthology.org Official leaderboard
14 NarSiL 70.3% Track A NarSiL aclanthology.org Official leaderboard
15 NCL&HKU-NarrSim 70.3% Track A NCL&HKU-NarrSim aclanthology.org Official leaderboard
16 DUTIR 69.8% Track A DUTIR aclanthology.org Official leaderboard
17 CophiWue 69.5% Track A CophiWue aclanthology.org Official leaderboard
18 CITD@UIT 69.3% Track A CITD@UIT aclanthology.org Official leaderboard
19 Narrative Team 69.3% Track B Narrative Team aclanthology.org Official leaderboard
20 ttda704 69.3% Track A ttda704 aclanthology.org Official leaderboard
21 Narrative Nexus 68.5% Track A Narrative Nexus aclanthology.org Official leaderboard
22 Comhis 68.3% Track A Comhis aclanthology.org Official leaderboard
23 harapalb 68.3% Track A harapalb aclanthology.org Official leaderboard
24 SoloSemantics 68% Track A SoloSemantics aclanthology.org Official leaderboard
25 schmerle 67.8% Track A schmerle aclanthology.org Official leaderboard
26 L3IRIT 65.8% Track A L3IRIT aclanthology.org Official leaderboard
27 MarSan 65.5% Track B MarSan aclanthology.org Official leaderboard
28 NLP-FSDM 65.5% Track A NLP-FSDM aclanthology.org Official leaderboard
29 Team UBSE 65.3% Track A Team UBSE aclanthology.org Official leaderboard
30 MoodMetric 65% Track A MoodMetric aclanthology.org Official leaderboard
31 PEU Lab 64.5% Track A PEU Lab aclanthology.org Official leaderboard
32 CUNI 64.3% Track B CUNI aclanthology.org Official leaderboard
33 ChulaNLP 63.5% Track A ChulaNLP aclanthology.org Official leaderboard
34 blue 62.5% Track A blue aclanthology.org Official leaderboard
35 Team HausaNLP 61.5% Track A Team HausaNLP aclanthology.org Official leaderboard
36 TFB 61.5% Track B TFB aclanthology.org Official leaderboard
37 Spinfo Cologne 60.3% Track A Spinfo Cologne aclanthology.org Official leaderboard
38 LIAAD INESCTEC 59.8% Track A LIAAD INESCTEC aclanthology.org Official leaderboard
39 Cryptix 59.5% Track A Cryptix aclanthology.org Official leaderboard
40 VerbaNexAI 59.3% Track B VerbaNexAI aclanthology.org Official leaderboard
41 CICL26 59% Track A CICL26 aclanthology.org Official leaderboard
42 Duluth 58.5% Track A Duluth aclanthology.org Official leaderboard
43 Mendel292 58% Track B Mendel292 aclanthology.org Official leaderboard
44 IIITH Boys 57.8% Track A IIITH Boys aclanthology.org Official leaderboard
45 Lacuna Inc. 57.3% Track A Lacuna Inc. aclanthology.org Official leaderboard
46 PLlama 55.5% Track A PLlama aclanthology.org Official leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Knowledge

All 332 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy. Seed1.8 92.3%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%

Data audit

All 8 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
  • Pass Independent runs Scores from the official leaderboard (46 entries).
  • Pass Models scored 46 entries on its leaderboard, none a model we track.
  • Note Newest score No scores yet.
  • Note Frontier models No models scored yet.
  • Note Headroom No scores yet.
Leaderboard import 3 pass
  • Pass Importer We read the board on narrative-similarity-task.github.io: 46 entries at the last read, none a model we track.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check