Benchmark Almanac

XACLE Challenge 2026

Sun_NPU_01 0.64 tops its leaderboard
Not in model cards

Models listen to generated audio and predict how closely it matches an English description. Organizers rank predictions by their correlation with listener scores.

Tokyo Metropolitan University used Qwen2.5 and BEATs and placed third among 18 teams. Four listeners rated each of the 3,000 test pairs.

Released
July 2025
Kind
Audio understanding and voice
Level
Everyday
Scored
Checked automatically
Answers
Prediction
Tasks
All public
Measures
Capability
Language
English
In and out
Audio, Text → Text
Interaction
Single turn
Items from
Generated
Refresh
One fixed set
Run by
A host
In views
Audio and speech
49
Entries on its own leaderboard, 0 models we track

Sun_NPU_01 tops the leaderboard with 0.64

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

  1. 1 Sun_NPU_01 0.64
  2. 2 Chunarkar_NTHU_01 0.64
  3. 3 Chunarkar_NTHU_03 0.64
  4. 4 Chunarkar_NTHU_04 0.64
  5. 5 Shiota_TMU_4 0.63
  6. 6 Huang_WHU_01 0.63
  7. 7 Shiota_TMU_3 0.63
  8. 8 Shiota_TMU_1 0.63
  9. 9 Huang_WHU_02 0.62
  10. 10 Chunarkar_NTHU_02 0.62
  11. 11 Guan_HEU_02 0.61
  12. 12 Takano_UTokyo_03 0.61
  13. 13 Takano_UTokyo_02 0.61
  14. 14 Takano_UTokyo_01 0.61
  15. 15 Takano_UTokyo_04 0.61
  16. 16 Huang_WHU_04 0.61
  17. 17 Guan_HEU_03 0.60
  18. 18 Huang_WHU_03 0.59
  19. 19 Sun_NPU_04 0.58
  20. 20 Huy_HCMUS_01 0.57
  21. 21 Nakagome_LINEWORKS_01 0.57
  22. 22 Niizumi_TMUNTT_3 0.57
  23. 23 Niizumi_TMUNTT_1 0.57
  24. 24 Sun_NPU_03 0.56
  25. 25 Kumar_TII_1 0.56
  26. 26 Niizumi_TMUNTT_4 0.56
  27. 27 Niizumi_TMUNTT_2 0.56
  28. 28 Shiota_TMU_2 0.56
  29. 29 Guan_HEU_01 0.55
  30. 30 Lim_CAU_01 0.55
  31. 31 Sun_NPU_02 0.54
  32. 32 Lim_CAU_02 0.54
  33. 33 Lim_CAU_04 0.53
  34. 34 Sun_xiaomi_01 0.53
  35. 35 Sun_xiaomi_03 0.53
  36. 36 Vedula_ASML_4 0.51
  37. 37 Lim_CAU_03 0.51
  38. 38 Vedula_ASML_1 0.50
  39. 39 Vedula_ASML_2 0.50
  40. 40 Vedula_ASML_3 0.48
  41. 41 lyu_alibaba_1 0.48
  42. 42 Pantelimon_NUSTPB_4 0.46
  43. 43 Pantelimon_NUSTPB_1 0.46
  44. 44 Kim_GIST_1 0.46
  45. 45 Pantelimon_NUSTPB_3 0.46
  46. 46 Pantelimon_NUSTPB_2 0.45
  47. 47 Baseline 0.33
  48. 48 Pikrakis_UPiraeus_1 0.31
  49. 49 hf_test_1 0.31

Source: Official XACLE Challenge 2026 leaderboard

#EntryScoreSettingSubmitted byDateSource
1 Sun_NPU_01 0.64 Sun_NPU Official leaderboard
2 Chunarkar_NTHU_01 0.64 Chunarkar_NTHU Official leaderboard
3 Chunarkar_NTHU_03 0.64 Chunarkar_NTHU Official leaderboard
4 Chunarkar_NTHU_04 0.64 Chunarkar_NTHU Official leaderboard
5 Shiota_TMU_4 0.63 Shiota_TMU Official leaderboard
6 Huang_WHU_01 0.63 Huang_WHU Official leaderboard
7 Shiota_TMU_3 0.63 Shiota_TMU Official leaderboard
8 Shiota_TMU_1 0.63 Shiota_TMU Official leaderboard
9 Huang_WHU_02 0.62 Huang_WHU Official leaderboard
10 Chunarkar_NTHU_02 0.62 Chunarkar_NTHU Official leaderboard
11 Guan_HEU_02 0.61 Guan_HEU Official leaderboard
12 Takano_UTokyo_03 0.61 Takano_UTokyo Official leaderboard
13 Takano_UTokyo_02 0.61 Takano_UTokyo Official leaderboard
14 Takano_UTokyo_01 0.61 Takano_UTokyo Official leaderboard
15 Takano_UTokyo_04 0.61 Takano_UTokyo Official leaderboard
16 Huang_WHU_04 0.61 Huang_WHU Official leaderboard
17 Guan_HEU_03 0.60 Guan_HEU Official leaderboard
18 Huang_WHU_03 0.59 Huang_WHU Official leaderboard
19 Sun_NPU_04 0.58 Sun_NPU Official leaderboard
20 Huy_HCMUS_01 0.57 Huy_HCMUS Official leaderboard
21 Nakagome_LINEWORKS_01 0.57 Nakagome_LINEWORKS Official leaderboard
22 Niizumi_TMUNTT_3 0.57 Niizumi_TMUNTT Official leaderboard
23 Niizumi_TMUNTT_1 0.57 Niizumi_TMUNTT Official leaderboard
24 Sun_NPU_03 0.56 Sun_NPU Official leaderboard
25 Kumar_TII_1 0.56 Kumar_TII Official leaderboard
26 Niizumi_TMUNTT_4 0.56 Niizumi_TMUNTT Official leaderboard
27 Niizumi_TMUNTT_2 0.56 Niizumi_TMUNTT Official leaderboard
28 Shiota_TMU_2 0.56 Shiota_TMU Official leaderboard
29 Guan_HEU_01 0.55 Guan_HEU Official leaderboard
30 Lim_CAU_01 0.55 Lim_CAU Official leaderboard
31 Sun_NPU_02 0.54 Sun_NPU Official leaderboard
32 Lim_CAU_02 0.54 Lim_CAU Official leaderboard
33 Lim_CAU_04 0.53 Lim_CAU Official leaderboard
34 Sun_xiaomi_01 0.53 Sun_xiaomi Official leaderboard
35 Sun_xiaomi_03 0.53 Sun_xiaomi Official leaderboard
36 Vedula_ASML_4 0.51 Vedula_ASML Official leaderboard
37 Lim_CAU_03 0.51 Lim_CAU Official leaderboard
38 Vedula_ASML_1 0.50 Vedula_ASML Official leaderboard
39 Vedula_ASML_2 0.50 Vedula_ASML Official leaderboard
40 Vedula_ASML_3 0.48 Vedula_ASML Official leaderboard
41 lyu_alibaba_1 0.48 lyu_alibaba Official leaderboard
42 Pantelimon_NUSTPB_4 0.46 Pantelimon_NUSTPB Official leaderboard
43 Pantelimon_NUSTPB_1 0.46 Pantelimon_NUSTPB Official leaderboard
44 Kim_GIST_1 0.46 Kim_GIST Official leaderboard
45 Pantelimon_NUSTPB_3 0.46 Pantelimon_NUSTPB Official leaderboard
46 Pantelimon_NUSTPB_2 0.45 Pantelimon_NUSTPB Official leaderboard
47 Baseline 0.33 Official leaderboard
48 Pikrakis_UPiraeus_1 0.31 Pikrakis_UPiraeus Official leaderboard
49 hf_test_1 0.31 hf_test Official leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Audio and video

All 695 in Audio and video
  • Video-MMMU In use 300 expert-level lecture videos with questions testing perception, comprehension and adaptation of knowledge. Accuracy. Qwen 3.8 Max 88.7%
  • LVBench In use Hours-long video understanding with multiple-choice questions on reasoning, grounding and summarization. Gemini 4 Argon 91.7%
  • MMVU In use 3,000 expert-annotated questions on videos across 27 disciplines. Multiple-choice and open-ended accuracy. Qwen 3.8 Max 82.4%
  • MLVU In use Multi-task long video understanding (minutes to hours) with multiple-choice and generation tasks. Accuracy. Qwen 3.8 Max 90.8%
  • FLEURS In use Models transcribe read speech in 102 languages built from FLoRes-101 sentences. Model cards usually report word error rate, so lower scores are better. Phi-4-multimodal 4.00%
  • VoiceBench In use Spoken instructions with speaker, environment and content variation. Voice assistant responses scored by GPT judge or rules. Qwen3.5-Omni-Plus 93.1%

Data audit

All 8 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
  • Pass Independent runs Scores from the official leaderboard (49 entries).
  • Pass Models scored 49 entries on its leaderboard, none a model we track.
  • Note Newest score No scores yet.
  • Note Frontier models No models scored yet.
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on xacle.org: 49 entries at the last read, none a model we track.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check