Benchmark Almanac

Open Japanese LLM Leaderboard

deep-analysis-research/Flux-Japanese-Qwen2.5-32B-Instruct-V1.0 0.74 tops its leaderboard QwQ-32B-Preview 0.67 leads the models we track
Not in model cards

Models perform Japanese language tasks drawn from more than 20 datasets.

LLM-jp and Hugging Face publish the board together.

Made by
LLM-jp and Hugging Face
Released
November 2024
Kind
Knowledge in many languages
Level
University
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
Japanese
In and out
Text → Text
Interaction
Single turn
Items from
Another benchmark
Refresh
One fixed set
Run by
Anyone
Strongest model tested
QwQ-32B #137 on the capability index
Last model added
May 2025
Scores refreshed
November 20th, 2026
418
Entries on its own leaderboard, 60 models we track

deep-analysis-research/Flux-Japanese-Qwen2.5-32B-Instruct-V1.0 tops the leaderboard with 0.74 QwQ-32B-Preview leads with 0.67 among models released by May 2025

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is QwQ-32B-Preview, at 0.67. This view shows only models we track, each at its best.

The newest model with a score here came out in May 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

418 entries. Scroll sideways for the rest.

  1. 0.74 deep-analysis-research/Flux-Japanese-Qwen2.5-32B-Instruct-V1.0
  2. 0.71 deep-analysis-research/D2IL-Japanese-Qwen2.5-32B-Instruct-v0.1
  3. 0.70 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B
  4. 0.70 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V5-70B
  5. 0.70 nitky/Llama-3.1-SuperSwallow-70B-Instruct-v0.1
  6. 0.70 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V4-70B
  7. 0.69 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V4-70B
  8. 0.69 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V2-70B
  9. 0.69 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V11-70B
  10. 0.69 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V3-70B
  11. 0.68 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V10-70B
  12. 0.68 allenai/Llama-3.1-Tulu-3-70B
  13. 0.68 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V5-70B
  14. 0.68 yellowtown/Japanese-Qwen2.5-14B-Instruct-V1
  15. 0.68 FINGU-AI/FINGU-2.5-instruct-32B-v1
  16. 0.68 karakuri-ai/karakuri-lm-32b-thinking-2501-exp
  17. 0.68 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-70B
  18. 0.68 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V2-70B
  19. 0.67 FINGU-AI/FINGU-2.5-instruct-70B
  20. 0.67 nitky/RoguePlanet-DeepSeek-R1-Qwen-32B-RP
  21. 0.67 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V9-70B
  22. 0.67 JungZoona/T3Q-qwen2.5-32b-v1.2-e2
  23. 0.67 QwQ-32B-Preview
  24. 0.67 cyberagent/Llama-3.1-70B-Japanese-Instruct-2407
  25. 0.67 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V1-70B
  26. 0.67 huihui-ai/QwQ-32B-Coder-Fusion-9010
  27. 0.67 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V12-70B
  28. 0.67 maldv/Awqward2.5-32B-Instruct
  29. 0.67 Saxo/Linkbricks-Horizon-AI-Japanese-Base-32B
  30. 0.67 Triangle104/DeepSky-T100
  31. 0.67 NovaSky-AI/Sky-T1-32B-Flash
  32. 0.66 meta-llama/Llama-3.1-70B-Instruct
  33. 0.66 rinna/qwen2.5-bakeneko-32b-instruct-v2
  34. 0.66 Sao10K/32B-Qwen2.5-Kunou-v1
  35. 0.66 Llama-3.1-Nemotron-70B-Instruct
  36. 0.66 NovaSky-AI/Sky-T1-32B-Preview
  37. 0.66 spow12/ChatWaifu_32B_reasoning
  38. 0.66 simplescaling/s1-32B
  39. 0.66 maldv/Qwenstein2.5-32B-Instruct
  40. 0.66 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V7-70B
  41. 0.66 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V6-70B
  42. 0.66 deepcogito/cogito-v1-preview-qwen-32B
  43. 0.66 shisa-ai/shisa-v2-qwen2.5-32b
  44. 0.66 zetasepic/Qwen2.5-32B-Instruct-abliterated-v2
  45. 0.66 arcee-ai/Virtuoso-Medium-v2
  46. 0.66 Sakalti/ultiima-32B
  47. 0.66 rombodawg/Rombos-LLM-V2.5-Qwen-32b
  48. 0.66 prithivMLmods/Galactic-Qwen-14B-Exp2
  49. 0.66 deep-analysis-research/test-qwen2.5-32b
  50. 0.66 Qwen/Qwen2.5-32B-Instruct
  51. 0.66 TIGER-Lab/Qwen2.5-32B-Instruct-CFT
  52. 0.66 Sakalti/ultiima-52B
  53. 0.65 Sorawiz/Qwen2.5-14B-Instinct
  54. 0.65 hotmailuser/RombosBeagle-v2beta-MGS-32B
  55. 0.65 AXCXEPT/Llama-3.1-70B-EZO-1.1-it
  56. 0.65 mergekit-community/mergekit-model_stock-odyqbix
  57. 0.65 Saxo/Linkbricks-Horizon-AI-Korean-Advanced-70B
  58. 0.65 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V5-70B
  59. 0.65 llm-jp/llm-jp-3.1-13b-instruct4
  60. 0.65 Sorawiz/Qwen2.5-14B-Instinct-Talk
  61. 0.65 Sakalti/Saka-14B
  62. 0.65 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V3-70B
  63. 0.65 mergekit-community/mergekit-model_stock-lkrglxd
  64. 0.65 prithivMLmods/Sombrero-Opus-14B-Elite6
  65. 0.65 mergekit-community/mergekit-model_stock-qndyhny
  66. 0.65 Sorawiz/Qwen2.5-14B-Instinct-RP
  67. 0.65 mergekit-community/mergekit-ties-dvjxpbu
  68. 0.65 suayptalha/Lamarckvergence-14B
  69. 0.65 mergekit-community/mergekit-ties-knfxihf
  70. 0.65 mergekit-community/mergekit-della-efwskwi
  71. 0.65 prithivMLmods/Messier-Opus-14B-Elite7
  72. 0.65 mergekit-community/mergekit-model_stock-fuvlumz
  73. 0.65 rinna/llama-3-youko-70b-instruct
  74. 0.65 mergekit-community/mergekit-slerp-ryfxivm
  75. 0.65 mergekit-community/mergekit-model_stock-rmxmzvo
  76. 0.65 mergekit-community/mergekit-model_stock-jlodpmg
  77. 0.65 sometimesanotion/Lamarck-14B-v0.7
  78. 0.65 CultriX/Qwen2.5-14B-Hyperionv4
  79. 0.65 prithivMLmods/Equuleus-Opus-14B-Exp
  80. 0.65 sthenno/tempesthenno-nuslerp-0124
  81. 0.65 sometimesanotion/Qwenvergence-14B-v12-Prose-DS
  82. 0.65 sometimesanotion/Lamarck-14B-v0.7-Fusion
  83. 0.65 sthenno-com/miscii-14b-0218
  84. 0.65 mergekit-community/mergekit-model_stock-dmnvnmc
  85. 0.65 Sao10K/14B-Qwen2.5-Freya-x1
  86. 0.65 sometimesanotion/Qwenvergence-14B-v13-Prose-DS
  87. 0.65 mergekit-community/mergekit-model_stock-jwxwwmz
  88. 0.65 sthenno/tempesthenno-icy-0130
  89. 0.65 Sakalti/ultiima-14B-v0.4
  90. 0.65 qihoo360/Light-R1-32B
  91. 0.65 sthenno/tempesthenno-sft-0314-stage1-ckpt50
  92. 0.65 CultriX/Qwen2.5-14B-Ultimav2
  93. 0.65 JungZoona/T3Q-qwen2.5-14b-v1.2-e2
  94. 0.65 mergekit-community/mergekit-model_stock-batsnqf
  95. 0.65 Sakalti/ultiima-14B-v0.2
  96. 0.65 sometimesanotion/Lamarck-14B-v0.6
  97. 0.65 Cran-May/tempmotacilla-cinerea-0308
  98. 0.65 hotmailuser/QwenSlerp2-14B
  99. 0.65 sometimesanotion/Qwenvergence-14B-v11
  100. 0.65 mergekit-community/mergekit-model_stock-skvbpno
  101. 0.65 sthenno/tempesthenno-ppo-ckpt40
  102. 0.65 maldv/Qwentile2.5-32B-Instruct
  103. 0.65 sthenno-com/miscii-14b-1225
  104. 0.65 Qwen2.5-14B-Instruct-1M
  105. 0.64 deepcogito/cogito-v1-preview-qwen-14B
  106. 0.64 aixonlab/Zara-14b-v1.2
  107. 0.64 Nohobby/Q2.5-Qwetiapin-32B
  108. 0.64 mergekit-community/mergekit-model_stock-zelysxr
  109. 0.64 jpacifico/Chocolatine-2-14B-Instruct-v2.0b3
  110. 0.64 jpacifico/Chocolatine-2-14B-Instruct-v2.0.3
  111. 0.64 JungZoona/T3Q-qwen2.5-14b-v1.0-e3
  112. 0.64 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V3-70B
  113. 0.64 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V2-70B
  114. 0.64 sometimesanotion/Qwenvergence-14B-v10
  115. 0.64 fluently-lm/FluentlyLM-Prinum
  116. 0.64 prithivMLmods/Alpha-UMa-Qwen-14B
  117. 0.64 Sao10K/14B-Qwen2.5-Kunou-v1
  118. 0.64 sometimesanotion/Qwen2.5-14B-Vimarckoso-v3
  119. 0.64 Qwen2.5-14B
  120. 0.64 arcee-ai/Virtuoso-Small
  121. 0.64 soob3123/amoral-cogito-14b
  122. 0.64 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V4-70B
  123. 0.64 arcee-ai/Virtuoso-Small-v2
  124. 0.64 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V1-32B
  125. 0.64 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V1-70B
  126. 0.64 rubenroy/Zurich-14B-GCv2-5m
  127. 0.64 mergekit-community/mergekit-model_stock-grrwjhl
  128. 0.64 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V8-70B
  129. 0.64 Ttimofeyka/Tissint-14B-v1.2-128k-RP
  130. 0.63 Cogito v1 Preview 70B
  131. 0.63 Saxo/Linkbricks-Horizon-AI-Avengers-V6-32B
  132. 0.63 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V7-70B
  133. 0.63 nicoboss/DeepSeek-R1-Distill-Qwen-32B-Uncensored
  134. 0.63 Saxo/Linkbricks-Horizon-AI-Avengers-V3-32B
  135. 0.63 SicariusSicariiStuff/Impish_QWEN_14B-1M
  136. 0.63 Sakalti/oxyge1-33B
  137. 0.63 Sakalti/Oxyge1-33B
  138. 0.63 hotmailuser/Deeepseek-QwenSlerp4-32B
  139. 0.63 TeamDelta/Re-ultiima-14B
  140. 0.63 YOYO-AI/Qwen2.5-14B-YOYO-Average
  141. 0.63 Saxo/Linkbricks-Horizon-AI-Avengers-V1-32B
  142. 0.63 CultriX/Qwen2.5-14B-HyperSeekv5
  143. 0.63 Qwen/Qwen2.5-14B-Instruct
  144. 0.63 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V3-70B
  145. 0.63 Sakalti/ultiima-14B
  146. 0.63 Llama 3-70B
  147. 0.63 netease-youdao/Confucius-o1-14B
  148. 0.63 tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4
  149. 0.63 nitky/EZO-QwQ-32B-Preview
  150. 0.63 meta-llama/Meta-Llama-3-70B-Instruct
  151. 0.62 DataPilot/SKYDRIVE-32B-v0.1
  152. 0.62 tanliboy/lambda-qwen2.5-14b-dpo-test
  153. 0.62 deep-analysis-research/test-qwen2.5-14b-wo-system-v1
  154. 0.62 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V6-70B
  155. 0.62 AXCXEPT/EZO-Qwen2.5-32B-Instruct
  156. 0.62 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-70B
  157. 0.62 nitky/Llama-3.3-FakeSwallow-70B-Instruct-v0.1
  158. 0.62 Saxo/Linkbricks-Horizon-AI-Avengers-V2-32B
  159. 0.62 tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.3
  160. 0.62 tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.1
  161. 0.62 OpenBuddy/openbuddy-qwq-32b-v24.1-200k
  162. 0.62 Qwen2.5-Coder-32B-Instruct
  163. 0.62 DataPilot/Arrival-32B-Instruct-v0.5
  164. 0.62 nitky/Llama-3.3-SuperSwallow-70B-Instruct-v0.1
  165. 0.62 rombodawg/Rombos-Coder-V2.5-Qwen-32b
  166. 0.62 wanlige/li-14b-v0.4-slerp0.1
  167. 0.62 nbeerbower/DeepSeek-R1-Qwen-lorablated-32B
  168. 0.61 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V5-70B
  169. 0.61 Llama 3.3 70B Instruct
  170. 0.61 Qwen/Qwen2-57B-A14B
  171. 0.61 shisa-ai/shisa-v2-mistral-small-24b
  172. 0.61 nitky/RoguePlanet-DeepSeek-R1-Qwen-32B
  173. 0.61 Saxo/Linkbricks-Horizon-AI-Avengers-V4-32B
  174. 0.61 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V8-70B
  175. 0.61 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V2-70B
  176. 0.61 Saxo/Linkbricks-Horizon-AI-Llama-3.3-Japanese-70B-sft-dpo-base
  177. 0.61 Saxo/Linkbricks-Horizon-AI-Avengers-V5-32B
  178. 0.61 Steelskull/L3.3-MS-Nevoria-70b
  179. 0.61 RDson/WomboCombo-R1-Coder-14B-Preview
  180. 0.61 Saxo/Linkbricks-Horizon-AI-Japanese-Base-70B
  181. 0.61 Qwen2.5-32B
  182. 0.61 shisa-ai/shisa-v2-llama3.3-70b
  183. 0.60 wanlige/li-14b-v0.4
  184. 0.60 Ba2han/QwQenSeek-coder
  185. 0.60 TeetouchQQ/model_mergev2
  186. 0.60 Azure99/Blossom-V6-32B
  187. 0.60 abeja/ABEJA-Qwen2.5-32b-Japanese-v1.0
  188. 0.60 cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese
  189. 0.60 abeja/ABEJA-Qwen2.5-32b-Japanese-v0.1
  190. 0.60 FINGU-AI/FINGU-2.5-instruct-32B
  191. 0.60 FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview
  192. 0.60 DreadPoor/Irix-12B-Model_Stock
  193. 0.59 RDson/CoderO1-DeepSeekR1-Coder-32B-Preview
  194. 0.59 spow12/ChatWaifu_v1.4
  195. 0.59 Llama 3.1-70B
  196. 0.59 Gen-Verse/ReasonFlux-F1
  197. 0.59 DeepSeek-R1-Distill-Qwen-32B
  198. 0.59 Saxo/Linkbricks-Horizon-AI-Llama-3.3-Japanese-70B-Advanced
  199. 0.59 NaniDAO/deepseek-r1-qwen-2.5-32B-ablated
  200. 0.59 llm-jp/llm-jp-3.1-13b
  201. 0.59 prithivMLmods/Qwen2.5-14B-DeepSeek-R1-1M
  202. 0.59 spow12/ChatWaifu_12B_v2.0
  203. 0.59 QwQ-32B
  204. 0.59 Saxo/Linkbricks-Horizon-AI-Korean-Superb-22B
  205. 0.58 Elizezen/Himeyuri-v0.1-12B
  206. 0.58 DeepSeek-R1-Distill-Llama-70B
  207. 0.58 mistralai/Mistral-Nemo-Instruct-2407
  208. 0.58 google/gemma-2-27b-it
  209. 0.58 anthracite-org/magnum-v2.5-12b-kto
  210. 0.58 Groq/Llama-3-Groq-70B-Tool-Use
  211. 0.58 karakuri-ai/karakuri-lm-8x7b-chat-v0.1
  212. 0.58 llm-jp/llm-jp-3.1-1.8b-instruct4
  213. 0.58 Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
  214. 0.58 Azure99/Blossom-V6-14B
  215. 0.58 huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated
  216. 0.58 shisa-ai/shisa-v2-mistral-nemo-12b
  217. 0.58 spow12/ChatWaifu_v2.0_22B
  218. 0.57 CohereForAI/aya-23-35B
  219. 0.57 Zhihu-ai/Zhi-writing-dsr1-14b
  220. 0.57 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-22B
  221. 0.57 agentica-org/DeepCoder-14B-Preview
  222. 0.57 nvidia/AceReason-Nemotron-14B
  223. 0.57 DeepSeek-R1-Distill-Qwen-14B
  224. 0.57 qingy2024/NaturalLM
  225. 0.57 qihoo360/TinyR1-32B-Preview
  226. 0.57 Qwen1.5-32B
  227. 0.57 rubenroy/Geneva-12B-GCv2-5m
  228. 0.56 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.2
  229. 0.56 cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese
  230. 0.56 Gen-Verse/ReasonFlux-F1-14B
  231. 0.56 cyberagent/calm3-22b-chat
  232. 0.56 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-27B
  233. 0.56 elyza/Llama-3-ELYZA-JP-8B
  234. 0.55 shisa-ai/shisa-v2-qwen2.5-7b
  235. 0.55 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3
  236. 0.55 shisa-ai/shisa-v1-llama3-70b
  237. 0.55 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.1
  238. 0.55 gmonsoon/SahabatAI-Lion-9B-TIES-v1
  239. 0.55 Qwen/Qwen2-57B-A14B-Instruct
  240. 0.55 tokyotech-llm/Llama-3.1-Swallow-8B-v0.1
  241. 0.55 Saxo/Linkbricks-Horizon-AI-Superb-27B
  242. 0.55 karakuri-ai/karakuri-lm-8x7b-instruct-v0.1
  243. 0.55 augmxnt/shisa-gamma-7b-v1
  244. 0.55 llm-jp/llm-jp-3-13b-instruct
  245. 0.55 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V7-70B
  246. 0.55 Reka Flash 3.1
  247. 0.54 Qwen1.5-14B
  248. 0.54 nbeerbower/Mistral-Nemo-Prism-12B-v2
  249. 0.54 rinna/llama-3-youko-8b-instruct
  250. 0.54 Kendamarron/Qwen2.5-7B-o1-ja-v0.1
  251. 0.54 lightblue/qwen2.5-7B-instruct-simpo
  252. 0.54 cyberagent/Mistral-Nemo-Japanese-Instruct-2408
  253. 0.54 meta-llama/Llama-2-70b-hf
  254. 0.54 Qwen/Qwen2.5-7B-Instruct-1M
  255. 0.54 AIDC-AI/Marco-o1
  256. 0.54 Qwen2.5-7B
  257. 0.54 tokyotech-llm/Llama-3.1-Swallow-8B-v0.2
  258. 0.54 Qwen2.5-Coder-14B-Instruct
  259. 0.54 rombodawg/Rombos-Coder-V2.5-Qwen-14b
  260. 0.54 Sakalti/deneb-v1-7b
  261. 0.54 Sakalti/Saka-7.6B
  262. 0.54 Saxo/Linkbricks-Horizon-AI-Korean-Advanced-56B
  263. 0.54 suayptalha/HomerCreativeAnvita-Mix-Qw7B
  264. 0.54 Qwen/Qwen1.5-32B-Chat
  265. 0.54 shisa-ai/shisa-v2-llama3.1-8b
  266. 0.54 DataPilot/Llama3.1-ArrowSE-v0.4
  267. 0.53 Azure99/Blossom-V6-7B
  268. 0.53 Nurture-intelligence/kEy-llama3.1-8b-v0.1
  269. 0.53 meta-llama/Llama-3.1-8B-Instruct
  270. 0.53 tokyo-electron-device-ai/llama3-tedllm-8b-v0-annealing
  271. 0.53 qihoo360/Light-R1-14B-DS
  272. 0.53 tokyotech-llm/Gemma-2-Llama-Swallow-27b-it-v0.1
  273. 0.53 Qwen/Qwen2.5-7B-Instruct
  274. 0.53 Qwen2-VL-7B-Instruct
  275. 0.53 AXCXEPT/EZO-Humanities-9B-gemma-2-it
  276. 0.53 fblgit/pancho-v1-qw25-3B-UNAMGS
  277. 0.53 AXCXEPT/EZO-Common-9B-gemma-2-it
  278. 0.53 deepcogito/cogito-v1-preview-llama-8B
  279. 0.53 Sakalti/sakalinear-1-1.0
  280. 0.52 AIDC-AI/Marco-LLM-GLO
  281. 0.52 rubenroy/Zurich-7B-GCv2-5m
  282. 0.52 Manual-Dataset-Creation-Project/Matsu-7B
  283. 0.52 Qwen2.5-3B
  284. 0.52 Qwen2-7B
  285. 0.52 google/gemma-2-9b-it
  286. 0.52 MaziyarPanahi/calme-3.2-instruct-3b
  287. 0.52 karakuri-ai/karakuri-lm-70b-chat-v0.1
  288. 0.52 Elizezen/Kudryavka-8B-alpha
  289. 0.52 Qwen2-7B-Instruct
  290. 0.52 Aya Expanse 8B
  291. 0.51 huihui-ai/DeepSeek-R1-Distill-Qwen-14B-abliterated-v2
  292. 0.51 Sakalti/Neptuno-Alpha
  293. 0.51 microsoft/Phi-3.5-mini-instruct
  294. 0.51 Aya Expanse 32B
  295. 0.51 Casual-Autopsy/Llama-3-VNTL-Yollisa-8B
  296. 0.51 huihui-ai/aya-expanse-32b-abliterated
  297. 0.51 Sakalti/Lunar-4B
  298. 0.50 ryota39/Tora-12B
  299. 0.50 SakanaAI/EvoLLM-JP-A-v1-7B
  300. 0.50 Qwen1.5-14B-Chat
  301. 0.50 meta-llama/Meta-Llama-3-8B-Instruct
  302. 0.49 llm-jp/llm-jp-3.1-1.8b
  303. 0.49 Granite-3.1-8B-Instruct
  304. 0.49 Llama 3.1-8B
  305. 0.49 Granite 3.2 8B Instruct
  306. 0.49 Llama 3-8B
  307. 0.49 llm-jp/llm-jp-3-13b
  308. 0.49 Qwen/Qwen2.5-Coder-7B-Instruct
  309. 0.48 CohereForAI/aya-23-8B
  310. 0.48 Qwen/Qwen2.5-3B-Instruct
  311. 0.48 meta-llama/Llama-2-70b-chat-hf
  312. 0.47 Qwen1.5-7B
  313. 0.47 tiiuae/Falcon3-10B-base
  314. 0.46 Rakuten/RakutenAI-7B-chat
  315. 0.46 deepcogito/cogito-v1-preview-llama-3B
  316. 0.46 llm-jp/llm-jp-3-3.7b-instruct
  317. 0.46 augmxnt/shisa-7b-v1
  318. 0.46 Granite-3.1-8B-Base
  319. 0.46 Elizezen/Phos-7B
  320. 0.45 Qwen2.5-1.5B
  321. 0.45 Qwen1.5-7B-Chat
  322. 0.45 Sakalti/Saba1.5-Pro
  323. 0.45 Sakalti/Saba2-Preview
  324. 0.45 Sakalti/tara-3.8B
  325. 0.45 Sakalti/Tara-3.8B-v1.1
  326. 0.45 jpacifico/Chocolatine-3B-Instruct-DPO-Revised
  327. 0.45 cognitivecomputations/Dolphin3.0-R1-Mistral-24B
  328. 0.45 Rakuten/RakutenAI-2.0-mini-instruct
  329. 0.45 rinna/gemma-2-baku-2b-it
  330. 0.45 Sakalti/Saka-1.5B
  331. 0.44 Qwen/Qwen2.5-1.5B-Instruct
  332. 0.44 Falcon3-10B-Instruct
  333. 0.44 Sakalti/SJT-1.5B-Alpha
  334. 0.44 Qwen/Qwen1.5-MoE-A2.7B
  335. 0.43 mobiuslabsgmbh/DeepSeek-R1-ReDistill-Llama3-8B-v1.1
  336. 0.43 Granite-3.1-2B-Instruct
  337. 0.43 TIGER-Lab/Qwen2.5-Math-7B-CFT
  338. 0.43 llm-jp/llm-jp-3-3.7b-instruct3
  339. 0.43 llm-jp/llm-jp-3-3.7b-instruct2
  340. 0.43 Saxo/Linkbricks-Horizon-AI-Korean-LLAMA3blend-8x8b
  341. 0.43 meta-llama/Llama-2-13b-hf
  342. 0.42 Llama 3.2 3B
  343. 0.42 tiiuae/Falcon3-7B-base
  344. 0.41 deepseek-ai/DeepSeek-R1-Distill-Llama-8B
  345. 0.41 meta-llama/Llama-3.2-3B-Instruct
  346. 0.41 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V6-70B
  347. 0.41 Granite-3.1-2B-Base
  348. 0.41 Qwen2-1.5B-Instruct
  349. 0.41 meta-llama/Llama-2-13b-chat-hf
  350. 0.41 Gemma 2 2B IT
  351. 0.40 Qwen/Qwen1.5-MoE-A2.7B-Chat
  352. 0.40 Elizezen/SniffyOtter-7B
  353. 0.40 Elizezen/Antler-7B
  354. 0.40 rubenroy/Zurich-1.5B-GCv2-5m
  355. 0.40 Qwen1.5-4B-Chat
  356. 0.39 DeepSeek-R1-Distill-Qwen-7B
  357. 0.39 llm-jp/llm-jp-3-1.8b-instruct
  358. 0.39 Qwen/Qwen1.5-4B
  359. 0.39 Qwen2-1.5B
  360. 0.39 mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-7B-v1.1
  361. 0.38 Gen-Verse/ReasonFlux-F1-7B
  362. 0.38 meta-llama/Llama-2-7b-chat-hf
  363. 0.38 Granite-3.1-3B-A800M-Instruct
  364. 0.38 Sakalti/SJT-2.4B-Alpha
  365. 0.38 weblab-GENIAC/Tanuki-8B-dpo-v1.0
  366. 0.38 llm-jp/llm-jp-3-3.7b
  367. 0.38 Saxo/Linkbricks-Horizon-AI-Superb-70B
  368. 0.37 meta-llama/Llama-2-7b-hf
  369. 0.37 Granite-3.1-3B-A800M-Base
  370. 0.35 Falcon3-7B-Instruct
  371. 0.35 cyberagent/calm2-7b-chat
  372. 0.34 Elizezen/Hameln-japanese-mistral-7B
  373. 0.34 llm-jp/llm-jp-3-1.8b
  374. 0.33 prithivMLmods/Qwen2.5-32B-DeepSeek-R1-Instruct
  375. 0.33 llm-jp/llm-jp-3-980m-instruct2
  376. 0.33 llm-jp/llm-jp-3-980m-instruct3
  377. 0.32 Qwen/Qwen2.5-Math-7B-Instruct
  378. 0.31 Qwen2.5-0.5B
  379. 0.31 SmolLM2 1.7B Instruct
  380. 0.31 Qwen/Qwen2.5-0.5B-Instruct
  381. 0.31 Sakalti/CodeSaka-7.6B
  382. 0.31 meta-llama/Llama-3.2-1B-Instruct
  383. 0.30 tiiuae/Falcon3-3B-base
  384. 0.30 Qwen/Qwen1.5-1.8B
  385. 0.30 Qwen/Qwen1.5-1.8B-Chat
  386. 0.29 Qwen2-0.5B
  387. 0.29 Qwen2-0.5B-Instruct
  388. 0.29 Granite-3.1-1B-A400M-Instruct
  389. 0.29 llm-jp/llm-jp-3-980m
  390. 0.29 yamatazen/Twilight-SCE-12B-v2
  391. 0.29 Gemma 2B
  392. 0.29 Gemma 1.1 2B IT
  393. 0.29 sail/Sailor2-1B-Chat
  394. 0.28 Falcon3-3B-Instruct
  395. 0.28 Llama 3.2 1B
  396. 0.27 google/gemma-2b-it
  397. 0.27 Granite-3.1-1B-A400M-Base
  398. 0.26 agentica-org/DeepScaleR-1.5B-Preview
  399. 0.26 agentica-org/DeepCoder-1.5B-Preview
  400. 0.26 DeepSeek-R1-Distill-Qwen-1.5B
  401. 0.25 llm-jp/llm-jp-3-440m-instruct2
  402. 0.25 llm-jp/llm-jp-3-440m-instruct3
  403. 0.24 Falcon3-1B-Instruct
  404. 0.24 llm-jp/llm-jp-3-440m
  405. 0.24 rsh345/llama3.1-8b-swallow-openmath-dare_ties-d5w5_d5w5
  406. 0.23 nvidia/OpenMath2-Llama3.1-8B
  407. 0.23 Qwen/Qwen1.5-0.5B
  408. 0.21 tiiuae/Falcon3-1B-base
  409. 0.21 Sakalti/MathSaka-7.6B
  410. 0.21 CjangCjengh/GaLLM-multi-14B-v0.1
  411. 0.19 CjangCjengh/GaLLM-14B-v0.2
  412. 0.19 Qwen/Qwen1.5-0.5B-Chat
  413. 0.18 llm-jp/llm-jp-3-150m
  414. 0.16 HuggingFaceTB/SmolLM2-135M-Instruct
  415. 0.15 google/gemma-7b-it
  416. 0.15 Gemma 1.1 7B IT
  417. 0.04 Gemma 7B
  418. 0.03 Sakalti/Saka-7.2B
  1. 1 deep-analysis-research/Flux-Japanese-Qwen2.5-32B-Instruct-V1.0 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.74
  2. 2 deep-analysis-research/D2IL-Japanese-Qwen2.5-32B-Instruct-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.71
  3. 3 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.70
  4. 4 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V5-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.70
  5. 5 nitky/Llama-3.1-SuperSwallow-70B-Instruct-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.70
  6. 6 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V4-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.70
  7. 7 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V4-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.69
  8. 8 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V2-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.69
  9. 9 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V11-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.69
  10. 10 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.69
  11. 11 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V10-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  12. 12 allenai/Llama-3.1-Tulu-3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  13. 13 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V5-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  14. 14 yellowtown/Japanese-Qwen2.5-14B-Instruct-V1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  15. 15 FINGU-AI/FINGU-2.5-instruct-32B-v1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  16. 16 karakuri-ai/karakuri-lm-32b-thinking-2501-exp 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  17. 17 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  18. 18 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V2-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.68
  19. 19 FINGU-AI/FINGU-2.5-instruct-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  20. 20 nitky/RoguePlanet-DeepSeek-R1-Qwen-32B-RP 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  21. 21 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V9-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  22. 22 JungZoona/T3Q-qwen2.5-32b-v1.2-e2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  23. 23 QwQ-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  24. 24 cyberagent/Llama-3.1-70B-Japanese-Instruct-2407 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  25. 25 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V1-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  26. 26 huihui-ai/QwQ-32B-Coder-Fusion-9010 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  27. 27 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V12-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  28. 28 maldv/Awqward2.5-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  29. 29 Saxo/Linkbricks-Horizon-AI-Japanese-Base-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  30. 30 Triangle104/DeepSky-T100 4-shot / float16 / main / vLLM v0.6.3.post1 0.67
  31. 31 NovaSky-AI/Sky-T1-32B-Flash 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  32. 32 meta-llama/Llama-3.1-70B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  33. 33 rinna/qwen2.5-bakeneko-32b-instruct-v2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  34. 34 Sao10K/32B-Qwen2.5-Kunou-v1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  35. 35 Llama-3.1-Nemotron-70B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  36. 36 NovaSky-AI/Sky-T1-32B-Preview 4-shot / float16 / main / vLLM v0.6.3.post1 0.66
  37. 37 spow12/ChatWaifu_32B_reasoning 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  38. 38 simplescaling/s1-32B 4-shot / float32 / main / vLLM v0.6.3.post1 0.66
  39. 39 maldv/Qwenstein2.5-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  40. 40 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V7-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  41. 41 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V6-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  42. 42 deepcogito/cogito-v1-preview-qwen-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  43. 43 shisa-ai/shisa-v2-qwen2.5-32b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  44. 44 zetasepic/Qwen2.5-32B-Instruct-abliterated-v2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  45. 45 arcee-ai/Virtuoso-Medium-v2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  46. 46 Sakalti/ultiima-32B 4-shot / float16 / main / vLLM v0.6.3.post1 0.66
  47. 47 rombodawg/Rombos-LLM-V2.5-Qwen-32b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  48. 48 prithivMLmods/Galactic-Qwen-14B-Exp2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  49. 49 deep-analysis-research/test-qwen2.5-32b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  50. 50 Qwen/Qwen2.5-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  51. 51 TIGER-Lab/Qwen2.5-32B-Instruct-CFT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  52. 52 Sakalti/ultiima-52B 4-shot / float16 / main / vLLM v0.6.3.post1 0.66
  53. 53 Sorawiz/Qwen2.5-14B-Instinct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  54. 54 hotmailuser/RombosBeagle-v2beta-MGS-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  55. 55 AXCXEPT/Llama-3.1-70B-EZO-1.1-it 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  56. 56 mergekit-community/mergekit-model_stock-odyqbix 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  57. 57 Saxo/Linkbricks-Horizon-AI-Korean-Advanced-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  58. 58 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V5-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  59. 59 llm-jp/llm-jp-3.1-13b-instruct4 0-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  60. 60 Sorawiz/Qwen2.5-14B-Instinct-Talk 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  61. 61 Sakalti/Saka-14B 4-shot / float16 / main / vLLM v0.6.3.post1 0.65
  62. 62 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  63. 63 mergekit-community/mergekit-model_stock-lkrglxd 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  64. 64 prithivMLmods/Sombrero-Opus-14B-Elite6 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  65. 65 mergekit-community/mergekit-model_stock-qndyhny 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  66. 66 Sorawiz/Qwen2.5-14B-Instinct-RP 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  67. 67 mergekit-community/mergekit-ties-dvjxpbu 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  68. 68 suayptalha/Lamarckvergence-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  69. 69 mergekit-community/mergekit-ties-knfxihf 4-shot / float16 / main / vLLM v0.6.3.post1 0.65
  70. 70 mergekit-community/mergekit-della-efwskwi 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  71. 71 prithivMLmods/Messier-Opus-14B-Elite7 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  72. 72 mergekit-community/mergekit-model_stock-fuvlumz 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  73. 73 rinna/llama-3-youko-70b-instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  74. 74 mergekit-community/mergekit-slerp-ryfxivm 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  75. 75 mergekit-community/mergekit-model_stock-rmxmzvo 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  76. 76 mergekit-community/mergekit-model_stock-jlodpmg 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  77. 77 sometimesanotion/Lamarck-14B-v0.7 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  78. 78 CultriX/Qwen2.5-14B-Hyperionv4 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  79. 79 prithivMLmods/Equuleus-Opus-14B-Exp 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  80. 80 sthenno/tempesthenno-nuslerp-0124 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  81. 81 sometimesanotion/Qwenvergence-14B-v12-Prose-DS 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  82. 82 sometimesanotion/Lamarck-14B-v0.7-Fusion 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  83. 83 sthenno-com/miscii-14b-0218 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  84. 84 mergekit-community/mergekit-model_stock-dmnvnmc 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  85. 85 Sao10K/14B-Qwen2.5-Freya-x1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  86. 86 sometimesanotion/Qwenvergence-14B-v13-Prose-DS 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  87. 87 mergekit-community/mergekit-model_stock-jwxwwmz 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  88. 88 sthenno/tempesthenno-icy-0130 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  89. 89 Sakalti/ultiima-14B-v0.4 4-shot / float16 / main / vLLM v0.6.3.post1 0.65
  90. 90 qihoo360/Light-R1-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  91. 91 sthenno/tempesthenno-sft-0314-stage1-ckpt50 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  92. 92 CultriX/Qwen2.5-14B-Ultimav2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  93. 93 JungZoona/T3Q-qwen2.5-14b-v1.2-e2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  94. 94 mergekit-community/mergekit-model_stock-batsnqf 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  95. 95 Sakalti/ultiima-14B-v0.2 4-shot / float16 / main / vLLM v0.6.3.post1 0.65
  96. 96 sometimesanotion/Lamarck-14B-v0.6 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  97. 97 Cran-May/tempmotacilla-cinerea-0308 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  98. 98 hotmailuser/QwenSlerp2-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  99. 99 sometimesanotion/Qwenvergence-14B-v11 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  100. 100 mergekit-community/mergekit-model_stock-skvbpno 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  101. 101 sthenno/tempesthenno-ppo-ckpt40 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  102. 102 maldv/Qwentile2.5-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  103. 103 sthenno-com/miscii-14b-1225 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  104. 104 Qwen2.5-14B-Instruct-1M 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  105. 105 deepcogito/cogito-v1-preview-qwen-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  106. 106 aixonlab/Zara-14b-v1.2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  107. 107 Nohobby/Q2.5-Qwetiapin-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  108. 108 mergekit-community/mergekit-model_stock-zelysxr 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  109. 109 jpacifico/Chocolatine-2-14B-Instruct-v2.0b3 4-shot / float16 / main / vLLM v0.6.3.post1 0.64
  110. 110 jpacifico/Chocolatine-2-14B-Instruct-v2.0.3 4-shot / float16 / main / vLLM v0.6.3.post1 0.64
  111. 111 JungZoona/T3Q-qwen2.5-14b-v1.0-e3 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  112. 112 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  113. 113 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V2-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  114. 114 sometimesanotion/Qwenvergence-14B-v10 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  115. 115 fluently-lm/FluentlyLM-Prinum 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  116. 116 prithivMLmods/Alpha-UMa-Qwen-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  117. 117 Sao10K/14B-Qwen2.5-Kunou-v1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  118. 118 sometimesanotion/Qwen2.5-14B-Vimarckoso-v3 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  119. 119 Qwen2.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  120. 120 arcee-ai/Virtuoso-Small 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  121. 121 soob3123/amoral-cogito-14b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  122. 122 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V4-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  123. 123 arcee-ai/Virtuoso-Small-v2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  124. 124 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V1-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  125. 125 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V1-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  126. 126 rubenroy/Zurich-14B-GCv2-5m 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  127. 127 mergekit-community/mergekit-model_stock-grrwjhl 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  128. 128 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V8-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  129. 129 Ttimofeyka/Tissint-14B-v1.2-128k-RP 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  130. 130 Cogito v1 Preview 70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  131. 131 Saxo/Linkbricks-Horizon-AI-Avengers-V6-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  132. 132 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V7-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  133. 133 nicoboss/DeepSeek-R1-Distill-Qwen-32B-Uncensored 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  134. 134 Saxo/Linkbricks-Horizon-AI-Avengers-V3-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  135. 135 SicariusSicariiStuff/Impish_QWEN_14B-1M 4-shot / float16 / main / vLLM v0.6.3.post1 0.63
  136. 136 Sakalti/oxyge1-33B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  137. 137 Sakalti/Oxyge1-33B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  138. 138 hotmailuser/Deeepseek-QwenSlerp4-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  139. 139 TeamDelta/Re-ultiima-14B 4-shot / float16 / main / vLLM v0.6.3.post1 0.63
  140. 140 YOYO-AI/Qwen2.5-14B-YOYO-Average 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  141. 141 Saxo/Linkbricks-Horizon-AI-Avengers-V1-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  142. 142 CultriX/Qwen2.5-14B-HyperSeekv5 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  143. 143 Qwen/Qwen2.5-14B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  144. 144 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  145. 145 Sakalti/ultiima-14B 4-shot / float16 / main / vLLM v0.6.3.post1 0.63
  146. 146 Llama 3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  147. 147 netease-youdao/Confucius-o1-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  148. 148 tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  149. 149 nitky/EZO-QwQ-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  150. 150 meta-llama/Meta-Llama-3-70B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  151. 151 DataPilot/SKYDRIVE-32B-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  152. 152 tanliboy/lambda-qwen2.5-14b-dpo-test 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  153. 153 deep-analysis-research/test-qwen2.5-14b-wo-system-v1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  154. 154 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V6-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  155. 155 AXCXEPT/EZO-Qwen2.5-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  156. 156 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  157. 157 nitky/Llama-3.3-FakeSwallow-70B-Instruct-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  158. 158 Saxo/Linkbricks-Horizon-AI-Avengers-V2-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  159. 159 tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.3 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  160. 160 tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  161. 161 OpenBuddy/openbuddy-qwq-32b-v24.1-200k 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  162. 162 Qwen2.5-Coder-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  163. 163 DataPilot/Arrival-32B-Instruct-v0.5 4-shot / float16 / main / vLLM v0.6.3.post1 0.62
  164. 164 nitky/Llama-3.3-SuperSwallow-70B-Instruct-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  165. 165 rombodawg/Rombos-Coder-V2.5-Qwen-32b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  166. 166 wanlige/li-14b-v0.4-slerp0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  167. 167 nbeerbower/DeepSeek-R1-Qwen-lorablated-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  168. 168 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V5-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  169. 169 Llama 3.3 70B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  170. 170 Qwen/Qwen2-57B-A14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  171. 171 shisa-ai/shisa-v2-mistral-small-24b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  172. 172 nitky/RoguePlanet-DeepSeek-R1-Qwen-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  173. 173 Saxo/Linkbricks-Horizon-AI-Avengers-V4-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  174. 174 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V8-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  175. 175 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V2-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  176. 176 Saxo/Linkbricks-Horizon-AI-Llama-3.3-Japanese-70B-sft-dpo-base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  177. 177 Saxo/Linkbricks-Horizon-AI-Avengers-V5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  178. 178 Steelskull/L3.3-MS-Nevoria-70b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  179. 179 RDson/WomboCombo-R1-Coder-14B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  180. 180 Saxo/Linkbricks-Horizon-AI-Japanese-Base-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  181. 181 Qwen2.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  182. 182 shisa-ai/shisa-v2-llama3.3-70b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  183. 183 wanlige/li-14b-v0.4 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  184. 184 Ba2han/QwQenSeek-coder 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  185. 185 TeetouchQQ/model_mergev2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  186. 186 Azure99/Blossom-V6-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  187. 187 abeja/ABEJA-Qwen2.5-32b-Japanese-v1.0 4-shot / float16 / main / vLLM v0.6.3.post1 0.60
  188. 188 cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  189. 189 abeja/ABEJA-Qwen2.5-32b-Japanese-v0.1 4-shot / float16 / main / vLLM v0.6.3.post1 0.60
  190. 190 FINGU-AI/FINGU-2.5-instruct-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  191. 191 FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  192. 192 DreadPoor/Irix-12B-Model_Stock 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.60
  193. 193 RDson/CoderO1-DeepSeekR1-Coder-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  194. 194 spow12/ChatWaifu_v1.4 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  195. 195 Llama 3.1-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  196. 196 Gen-Verse/ReasonFlux-F1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  197. 197 DeepSeek-R1-Distill-Qwen-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  198. 198 Saxo/Linkbricks-Horizon-AI-Llama-3.3-Japanese-70B-Advanced 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  199. 199 NaniDAO/deepseek-r1-qwen-2.5-32B-ablated 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  200. 200 llm-jp/llm-jp-3.1-13b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  201. 201 prithivMLmods/Qwen2.5-14B-DeepSeek-R1-1M 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  202. 202 spow12/ChatWaifu_12B_v2.0 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  203. 203 QwQ-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  204. 204 Saxo/Linkbricks-Horizon-AI-Korean-Superb-22B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  205. 205 Elizezen/Himeyuri-v0.1-12B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  206. 206 DeepSeek-R1-Distill-Llama-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  207. 207 mistralai/Mistral-Nemo-Instruct-2407 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  208. 208 google/gemma-2-27b-it 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  209. 209 anthracite-org/magnum-v2.5-12b-kto 4-shot / float16 / main / vLLM v0.6.3.post1 0.58
  210. 210 Groq/Llama-3-Groq-70B-Tool-Use 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  211. 211 karakuri-ai/karakuri-lm-8x7b-chat-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  212. 212 llm-jp/llm-jp-3.1-1.8b-instruct4 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  213. 213 Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  214. 214 Azure99/Blossom-V6-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  215. 215 huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  216. 216 shisa-ai/shisa-v2-mistral-nemo-12b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  217. 217 spow12/ChatWaifu_v2.0_22B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  218. 218 CohereForAI/aya-23-35B 4-shot / float16 / main / vLLM v0.6.3.post1 0.57
  219. 219 Zhihu-ai/Zhi-writing-dsr1-14b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  220. 220 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-22B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  221. 221 agentica-org/DeepCoder-14B-Preview 4-shot / float32 / main / vLLM v0.6.3.post1 0.57
  222. 222 nvidia/AceReason-Nemotron-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  223. 223 DeepSeek-R1-Distill-Qwen-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  224. 224 qingy2024/NaturalLM 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  225. 225 qihoo360/TinyR1-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  226. 226 Qwen1.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  227. 227 rubenroy/Geneva-12B-GCv2-5m 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  228. 228 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.56
  229. 229 cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.56
  230. 230 Gen-Verse/ReasonFlux-F1-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.56
  231. 231 cyberagent/calm3-22b-chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.56
  232. 232 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-27B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.56
  233. 233 elyza/Llama-3-ELYZA-JP-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.56
  234. 234 shisa-ai/shisa-v2-qwen2.5-7b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  235. 235 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  236. 236 shisa-ai/shisa-v1-llama3-70b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  237. 237 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  238. 238 gmonsoon/SahabatAI-Lion-9B-TIES-v1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  239. 239 Qwen/Qwen2-57B-A14B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  240. 240 tokyotech-llm/Llama-3.1-Swallow-8B-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  241. 241 Saxo/Linkbricks-Horizon-AI-Superb-27B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  242. 242 karakuri-ai/karakuri-lm-8x7b-instruct-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  243. 243 augmxnt/shisa-gamma-7b-v1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  244. 244 llm-jp/llm-jp-3-13b-instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  245. 245 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V7-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  246. 246 Reka Flash 3.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  247. 247 Qwen1.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  248. 248 nbeerbower/Mistral-Nemo-Prism-12B-v2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  249. 249 rinna/llama-3-youko-8b-instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  250. 250 Kendamarron/Qwen2.5-7B-o1-ja-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  251. 251 lightblue/qwen2.5-7B-instruct-simpo 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  252. 252 cyberagent/Mistral-Nemo-Japanese-Instruct-2408 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  253. 253 meta-llama/Llama-2-70b-hf 4-shot / float16 / main / vLLM v0.6.3.post1 0.54
  254. 254 Qwen/Qwen2.5-7B-Instruct-1M 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  255. 255 AIDC-AI/Marco-o1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  256. 256 Qwen2.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  257. 257 tokyotech-llm/Llama-3.1-Swallow-8B-v0.2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  258. 258 Qwen2.5-Coder-14B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  259. 259 rombodawg/Rombos-Coder-V2.5-Qwen-14b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  260. 260 Sakalti/deneb-v1-7b 4-shot / float16 / main / vLLM v0.6.3.post1 0.54
  261. 261 Sakalti/Saka-7.6B 4-shot / float16 / main / vLLM v0.6.3.post1 0.54
  262. 262 Saxo/Linkbricks-Horizon-AI-Korean-Advanced-56B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  263. 263 suayptalha/HomerCreativeAnvita-Mix-Qw7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  264. 264 Qwen/Qwen1.5-32B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  265. 265 shisa-ai/shisa-v2-llama3.1-8b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  266. 266 DataPilot/Llama3.1-ArrowSE-v0.4 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  267. 267 Azure99/Blossom-V6-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  268. 268 Nurture-intelligence/kEy-llama3.1-8b-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  269. 269 meta-llama/Llama-3.1-8B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  270. 270 tokyo-electron-device-ai/llama3-tedllm-8b-v0-annealing 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  271. 271 qihoo360/Light-R1-14B-DS 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  272. 272 tokyotech-llm/Gemma-2-Llama-Swallow-27b-it-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  273. 273 Qwen/Qwen2.5-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  274. 274 Qwen2-VL-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  275. 275 AXCXEPT/EZO-Humanities-9B-gemma-2-it 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  276. 276 fblgit/pancho-v1-qw25-3B-UNAMGS 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  277. 277 AXCXEPT/EZO-Common-9B-gemma-2-it 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  278. 278 deepcogito/cogito-v1-preview-llama-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  279. 279 Sakalti/sakalinear-1-1.0 4-shot / float16 / main / vLLM v0.6.3.post1 0.53
  280. 280 AIDC-AI/Marco-LLM-GLO 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  281. 281 rubenroy/Zurich-7B-GCv2-5m 4-shot / float16 / main / vLLM v0.6.3.post1 0.52
  282. 282 Manual-Dataset-Creation-Project/Matsu-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  283. 283 Qwen2.5-3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  284. 284 Qwen2-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  285. 285 google/gemma-2-9b-it 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  286. 286 MaziyarPanahi/calme-3.2-instruct-3b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  287. 287 karakuri-ai/karakuri-lm-70b-chat-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  288. 288 Elizezen/Kudryavka-8B-alpha 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  289. 289 Qwen2-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  290. 290 Aya Expanse 8B 4-shot / float16 / main / vLLM v0.6.3.post1 0.52
  291. 291 huihui-ai/DeepSeek-R1-Distill-Qwen-14B-abliterated-v2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.51
  292. 292 Sakalti/Neptuno-Alpha 4-shot / float16 / main / vLLM v0.6.3.post1 0.51
  293. 293 microsoft/Phi-3.5-mini-instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.51
  294. 294 Aya Expanse 32B 4-shot / float16 / main / vLLM v0.6.3.post1 0.51
  295. 295 Casual-Autopsy/Llama-3-VNTL-Yollisa-8B 4-shot / float32 / main / vLLM v0.6.3.post1 0.51
  296. 296 huihui-ai/aya-expanse-32b-abliterated 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.51
  297. 297 Sakalti/Lunar-4B 4-shot / float16 / main / vLLM v0.6.3.post1 0.51
  298. 298 ryota39/Tora-12B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.50
  299. 299 SakanaAI/EvoLLM-JP-A-v1-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.50
  300. 300 Qwen1.5-14B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.50
  301. 301 meta-llama/Meta-Llama-3-8B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.50
  302. 302 llm-jp/llm-jp-3.1-1.8b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  303. 303 Granite-3.1-8B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  304. 304 Llama 3.1-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  305. 305 Granite 3.2 8B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  306. 306 Llama 3-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  307. 307 llm-jp/llm-jp-3-13b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  308. 308 Qwen/Qwen2.5-Coder-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  309. 309 CohereForAI/aya-23-8B 4-shot / float16 / main / vLLM v0.6.3.post1 0.48
  310. 310 Qwen/Qwen2.5-3B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.48
  311. 311 meta-llama/Llama-2-70b-chat-hf 4-shot / float16 / main / vLLM v0.6.3.post1 0.48
  312. 312 Qwen1.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.47
  313. 313 tiiuae/Falcon3-10B-base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.47
  314. 314 Rakuten/RakutenAI-7B-chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.46
  315. 315 deepcogito/cogito-v1-preview-llama-3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.46
  316. 316 llm-jp/llm-jp-3-3.7b-instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.46
  317. 317 augmxnt/shisa-7b-v1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.46
  318. 318 Granite-3.1-8B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.46
  319. 319 Elizezen/Phos-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.46
  320. 320 Qwen2.5-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.45
  321. 321 Qwen1.5-7B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.45
  322. 322 Sakalti/Saba1.5-Pro 4-shot / float16 / main / vLLM v0.6.3.post1 0.45
  323. 323 Sakalti/Saba2-Preview 4-shot / float16 / main / vLLM v0.6.3.post1 0.45
  324. 324 Sakalti/tara-3.8B 4-shot / float16 / main / vLLM v0.6.3.post1 0.45
  325. 325 Sakalti/Tara-3.8B-v1.1 4-shot / float16 / main / vLLM v0.6.3.post1 0.45
  326. 326 jpacifico/Chocolatine-3B-Instruct-DPO-Revised 4-shot / float16 / main / vLLM v0.6.3.post1 0.45
  327. 327 cognitivecomputations/Dolphin3.0-R1-Mistral-24B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.45
  328. 328 Rakuten/RakutenAI-2.0-mini-instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.45
  329. 329 rinna/gemma-2-baku-2b-it 4-shot / float16 / main / vLLM v0.6.3.post1 0.45
  330. 330 Sakalti/Saka-1.5B 4-shot / float16 / main / vLLM v0.6.3.post1 0.45
  331. 331 Qwen/Qwen2.5-1.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.44
  332. 332 Falcon3-10B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.44
  333. 333 Sakalti/SJT-1.5B-Alpha 4-shot / float16 / main / vLLM v0.6.3.post1 0.44
  334. 334 Qwen/Qwen1.5-MoE-A2.7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.44
  335. 335 mobiuslabsgmbh/DeepSeek-R1-ReDistill-Llama3-8B-v1.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.43
  336. 336 Granite-3.1-2B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.43
  337. 337 TIGER-Lab/Qwen2.5-Math-7B-CFT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.43
  338. 338 llm-jp/llm-jp-3-3.7b-instruct3 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.43
  339. 339 llm-jp/llm-jp-3-3.7b-instruct2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.43
  340. 340 Saxo/Linkbricks-Horizon-AI-Korean-LLAMA3blend-8x8b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.43
  341. 341 meta-llama/Llama-2-13b-hf 4-shot / float16 / main / vLLM v0.6.3.post1 0.43
  342. 342 Llama 3.2 3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.42
  343. 343 tiiuae/Falcon3-7B-base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.42
  344. 344 deepseek-ai/DeepSeek-R1-Distill-Llama-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  345. 345 meta-llama/Llama-3.2-3B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  346. 346 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V6-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  347. 347 Granite-3.1-2B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  348. 348 Qwen2-1.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  349. 349 meta-llama/Llama-2-13b-chat-hf 4-shot / float16 / main / vLLM v0.6.3.post1 0.41
  350. 350 Gemma 2 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  351. 351 Qwen/Qwen1.5-MoE-A2.7B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.40
  352. 352 Elizezen/SniffyOtter-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.40
  353. 353 Elizezen/Antler-7B 4-shot / float16 / main / vLLM v0.6.3.post1 0.40
  354. 354 rubenroy/Zurich-1.5B-GCv2-5m 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.40
  355. 355 Qwen1.5-4B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.40
  356. 356 DeepSeek-R1-Distill-Qwen-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.39
  357. 357 llm-jp/llm-jp-3-1.8b-instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.39
  358. 358 Qwen/Qwen1.5-4B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.39
  359. 359 Qwen2-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.39
  360. 360 mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-7B-v1.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.39
  361. 361 Gen-Verse/ReasonFlux-F1-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.38
  362. 362 meta-llama/Llama-2-7b-chat-hf 4-shot / float16 / main / vLLM v0.6.3.post1 0.38
  363. 363 Granite-3.1-3B-A800M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.38
  364. 364 Sakalti/SJT-2.4B-Alpha 4-shot / float16 / main / vLLM v0.6.3.post1 0.38
  365. 365 weblab-GENIAC/Tanuki-8B-dpo-v1.0 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.38
  366. 366 llm-jp/llm-jp-3-3.7b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.38
  367. 367 Saxo/Linkbricks-Horizon-AI-Superb-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.38
  368. 368 meta-llama/Llama-2-7b-hf 4-shot / float16 / main / vLLM v0.6.3.post1 0.37
  369. 369 Granite-3.1-3B-A800M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.37
  370. 370 Falcon3-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.35
  371. 371 cyberagent/calm2-7b-chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.35
  372. 372 Elizezen/Hameln-japanese-mistral-7B 4-shot / float16 / main / vLLM v0.6.3.post1 0.34
  373. 373 llm-jp/llm-jp-3-1.8b 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.34
  374. 374 prithivMLmods/Qwen2.5-32B-DeepSeek-R1-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.33
  375. 375 llm-jp/llm-jp-3-980m-instruct2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.33
  376. 376 llm-jp/llm-jp-3-980m-instruct3 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.33
  377. 377 Qwen/Qwen2.5-Math-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.32
  378. 378 Qwen2.5-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.31
  379. 379 SmolLM2 1.7B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.31
  380. 380 Qwen/Qwen2.5-0.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.31
  381. 381 Sakalti/CodeSaka-7.6B 4-shot / float16 / main / vLLM v0.6.3.post1 0.31
  382. 382 meta-llama/Llama-3.2-1B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.31
  383. 383 tiiuae/Falcon3-3B-base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.30
  384. 384 Qwen/Qwen1.5-1.8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.30
  385. 385 Qwen/Qwen1.5-1.8B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.30
  386. 386 Qwen2-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  387. 387 Qwen2-0.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  388. 388 Granite-3.1-1B-A400M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  389. 389 llm-jp/llm-jp-3-980m 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  390. 390 yamatazen/Twilight-SCE-12B-v2 0-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  391. 391 Gemma 2B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  392. 392 Gemma 1.1 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  393. 393 sail/Sailor2-1B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  394. 394 Falcon3-3B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.28
  395. 395 Llama 3.2 1B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.28
  396. 396 google/gemma-2b-it 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.27
  397. 397 Granite-3.1-1B-A400M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.27
  398. 398 agentica-org/DeepScaleR-1.5B-Preview 4-shot / float32 / main / vLLM v0.6.3.post1 0.26
  399. 399 agentica-org/DeepCoder-1.5B-Preview 4-shot / float32 / main / vLLM v0.6.3.post1 0.26
  400. 400 DeepSeek-R1-Distill-Qwen-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.26
  401. 401 llm-jp/llm-jp-3-440m-instruct2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.25
  402. 402 llm-jp/llm-jp-3-440m-instruct3 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.25
  403. 403 Falcon3-1B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.24
  404. 404 llm-jp/llm-jp-3-440m 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.24
  405. 405 rsh345/llama3.1-8b-swallow-openmath-dare_ties-d5w5_d5w5 4-shot / float16 / main / vLLM v0.6.3.post1 0.24
  406. 406 nvidia/OpenMath2-Llama3.1-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.23
  407. 407 Qwen/Qwen1.5-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.23
  408. 408 tiiuae/Falcon3-1B-base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.21
  409. 409 Sakalti/MathSaka-7.6B 4-shot / float16 / main / vLLM v0.6.3.post1 0.21
  410. 410 CjangCjengh/GaLLM-multi-14B-v0.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.21
  411. 411 CjangCjengh/GaLLM-14B-v0.2 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.19
  412. 412 Qwen/Qwen1.5-0.5B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.19
  413. 413 llm-jp/llm-jp-3-150m 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.18
  414. 414 HuggingFaceTB/SmolLM2-135M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.16
  415. 415 google/gemma-7b-it 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.15
  416. 416 Gemma 1.1 7B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.15
  417. 417 Gemma 7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 0.04
  418. 418 Sakalti/Saka-7.2B 0-shot / float16 / main / vLLM v0.6.3.post1 0.03

Source: Official Open Japanese LLM Leaderboard leaderboard

  1. 1 QwQ-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.67
  2. 2 Llama-3.1-Nemotron-70B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.66
  3. 3 Qwen2.5-14B-Instruct-1M 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.65
  4. 4 Qwen2.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.64
  5. 5 Cogito v1 Preview 70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  6. 6 Llama 3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.63
  7. 7 Qwen2.5-Coder-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.62
  8. 8 Llama 3.3 70B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  9. 9 Qwen2.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.61
  10. 10 Llama 3.1-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  11. 11 DeepSeek-R1-Distill-Qwen-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  12. 12 QwQ-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.59
  13. 13 DeepSeek-R1-Distill-Llama-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.58
  14. 14 DeepSeek-R1-Distill-Qwen-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  15. 15 Qwen1.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.57
  16. 16 Reka Flash 3.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.55
  17. 17 Qwen1.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  18. 18 Qwen2.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  19. 19 Qwen2.5-Coder-14B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.54
  20. 20 Qwen2-VL-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.53
  21. 21 Qwen2.5-3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  22. 22 Qwen2-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  23. 23 Qwen2-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.52
  24. 24 Aya Expanse 8B 4-shot / float16 / main / vLLM v0.6.3.post1 0.52
  25. 25 Aya Expanse 32B 4-shot / float16 / main / vLLM v0.6.3.post1 0.51
  26. 26 Qwen1.5-14B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.50
  27. 27 Granite-3.1-8B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  28. 28 Llama 3.1-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  29. 29 Granite 3.2 8B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  30. 30 Llama 3-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.49
  31. 31 Qwen1.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.47
  32. 32 Granite-3.1-8B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.46
  33. 33 Qwen2.5-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.45
  34. 34 Qwen1.5-7B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.45
  35. 35 Falcon3-10B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.44
  36. 36 Granite-3.1-2B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.43
  37. 37 Llama 3.2 3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.42
  38. 38 Granite-3.1-2B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  39. 39 Qwen2-1.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  40. 40 Gemma 2 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.41
  41. 41 Qwen1.5-4B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.40
  42. 42 DeepSeek-R1-Distill-Qwen-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.39
  43. 43 Qwen2-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.39
  44. 44 Granite-3.1-3B-A800M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.38
  45. 45 Granite-3.1-3B-A800M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.37
  46. 46 Falcon3-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.35
  47. 47 Qwen2.5-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.31
  48. 48 SmolLM2 1.7B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.31
  49. 49 Qwen2-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  50. 50 Qwen2-0.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  51. 51 Granite-3.1-1B-A400M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  52. 52 Gemma 2B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  53. 53 Gemma 1.1 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.29
  54. 54 Falcon3-3B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.28
  55. 55 Llama 3.2 1B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.28
  56. 56 Granite-3.1-1B-A400M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.27
  57. 57 DeepSeek-R1-Distill-Qwen-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.26
  58. 58 Falcon3-1B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.24
  59. 59 Gemma 1.1 7B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 0.15
  60. 60 Gemma 7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 0.04

Source: Official Open Japanese LLM Leaderboard leaderboard

#EntryScoreSettingDateLinkSource
1 deep-analysis-research/Flux-Japanese-Qwen2.5-32B-Instruct-V1.0 32.8B 0.74 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
2 deep-analysis-research/D2IL-Japanese-Qwen2.5-32B-Instruct-v0.1 32.8B 0.71 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
3 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V4-70B 70.6B 0.70 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
4 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V5-70B 70.6B 0.70 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
5 nitky/Llama-3.1-SuperSwallow-70B-Instruct-v0.1 70.6B 0.70 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
6 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V4-70B 70.6B 0.70 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
7 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V4-70B 70.6B 0.69 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
8 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V2-70B 70.6B 0.69 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
9 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V11-70B 70.6B 0.69 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
10 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V3-70B 70.6B 0.69 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
11 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V10-70B 70.6B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
12 allenai/Llama-3.1-Tulu-3-70B 70.6B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
13 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V5-70B 70.6B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
14 yellowtown/Japanese-Qwen2.5-14B-Instruct-V1 14.8B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
15 FINGU-AI/FINGU-2.5-instruct-32B-v1 32.8B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
16 karakuri-ai/karakuri-lm-32b-thinking-2501-exp 32.8B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
17 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-70B 70.6B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
18 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V2-70B 70.6B 0.68 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
19 FINGU-AI/FINGU-2.5-instruct-70B 70.6B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
20 nitky/RoguePlanet-DeepSeek-R1-Qwen-32B-RP 32.8B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
21 Saxo/Linkbricks-Horizon-AI-Japanese-Avdvanced-V9-70B 70.6B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
22 JungZoona/T3Q-qwen2.5-32b-v1.2-e2 32.8B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
23 QwQ-32B-Preview 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Nov 2024 huggingface.co Official leaderboard
24 cyberagent/Llama-3.1-70B-Japanese-Instruct-2407 70.6B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
25 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V1-70B 70.6B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
26 huihui-ai/QwQ-32B-Coder-Fusion-9010 32.8B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
27 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V12-70B 70.6B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
28 maldv/Awqward2.5-32B-Instruct 32.8B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
29 Saxo/Linkbricks-Horizon-AI-Japanese-Base-32B 32.8B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
30 Triangle104/DeepSky-T100 32.8B 0.67 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
31 NovaSky-AI/Sky-T1-32B-Flash 32.8B 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
32 meta-llama/Llama-3.1-70B-Instruct 70.6B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
33 rinna/qwen2.5-bakeneko-32b-instruct-v2 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
34 Sao10K/32B-Qwen2.5-Kunou-v1 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
35 Llama-3.1-Nemotron-70B-Instruct 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
36 NovaSky-AI/Sky-T1-32B-Preview 32.8B 0.66 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
37 spow12/ChatWaifu_32B_reasoning 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
38 simplescaling/s1-32B 32.8B 0.66 4-shot / float32 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
39 maldv/Qwenstein2.5-32B-Instruct 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
40 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V7-70B 70.6B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
41 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V6-70B 70.6B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
42 deepcogito/cogito-v1-preview-qwen-32B 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
43 shisa-ai/shisa-v2-qwen2.5-32b 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
44 zetasepic/Qwen2.5-32B-Instruct-abliterated-v2 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
45 arcee-ai/Virtuoso-Medium-v2 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
46 Sakalti/ultiima-32B 32.8B 0.66 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
47 rombodawg/Rombos-LLM-V2.5-Qwen-32b 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
48 prithivMLmods/Galactic-Qwen-14B-Exp2 14.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
49 deep-analysis-research/test-qwen2.5-32b 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
50 Qwen/Qwen2.5-32B-Instruct 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
51 TIGER-Lab/Qwen2.5-32B-Instruct-CFT 32.8B 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
52 Sakalti/ultiima-52B 52.3B 0.66 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
53 Sorawiz/Qwen2.5-14B-Instinct 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
54 hotmailuser/RombosBeagle-v2beta-MGS-32B 32.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
55 AXCXEPT/Llama-3.1-70B-EZO-1.1-it 70.6B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
56 mergekit-community/mergekit-model_stock-odyqbix 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
57 Saxo/Linkbricks-Horizon-AI-Korean-Advanced-70B 70.6B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
58 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V5-70B 70.6B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
59 llm-jp/llm-jp-3.1-13b-instruct4 13.7B 0.65 0-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
60 Sorawiz/Qwen2.5-14B-Instinct-Talk 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
61 Sakalti/Saka-14B 14.8B 0.65 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
62 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V3-70B 70.6B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
63 mergekit-community/mergekit-model_stock-lkrglxd 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
64 prithivMLmods/Sombrero-Opus-14B-Elite6 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
65 mergekit-community/mergekit-model_stock-qndyhny 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
66 Sorawiz/Qwen2.5-14B-Instinct-RP 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
67 mergekit-community/mergekit-ties-dvjxpbu 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
68 suayptalha/Lamarckvergence-14B 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
69 mergekit-community/mergekit-ties-knfxihf 14.8B 0.65 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
70 mergekit-community/mergekit-della-efwskwi 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
71 prithivMLmods/Messier-Opus-14B-Elite7 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
72 mergekit-community/mergekit-model_stock-fuvlumz 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
73 rinna/llama-3-youko-70b-instruct 70.6B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
74 mergekit-community/mergekit-slerp-ryfxivm 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
75 mergekit-community/mergekit-model_stock-rmxmzvo 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
76 mergekit-community/mergekit-model_stock-jlodpmg 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
77 sometimesanotion/Lamarck-14B-v0.7 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
78 CultriX/Qwen2.5-14B-Hyperionv4 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
79 prithivMLmods/Equuleus-Opus-14B-Exp 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
80 sthenno/tempesthenno-nuslerp-0124 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
81 sometimesanotion/Qwenvergence-14B-v12-Prose-DS 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
82 sometimesanotion/Lamarck-14B-v0.7-Fusion 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
83 sthenno-com/miscii-14b-0218 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
84 mergekit-community/mergekit-model_stock-dmnvnmc 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
85 Sao10K/14B-Qwen2.5-Freya-x1 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
86 sometimesanotion/Qwenvergence-14B-v13-Prose-DS 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
87 mergekit-community/mergekit-model_stock-jwxwwmz 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
88 sthenno/tempesthenno-icy-0130 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
89 Sakalti/ultiima-14B-v0.4 14.8B 0.65 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
90 qihoo360/Light-R1-32B 32.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
91 sthenno/tempesthenno-sft-0314-stage1-ckpt50 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
92 CultriX/Qwen2.5-14B-Ultimav2 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
93 JungZoona/T3Q-qwen2.5-14b-v1.2-e2 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
94 mergekit-community/mergekit-model_stock-batsnqf 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
95 Sakalti/ultiima-14B-v0.2 14.8B 0.65 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
96 sometimesanotion/Lamarck-14B-v0.6 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
97 Cran-May/tempmotacilla-cinerea-0308 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
98 hotmailuser/QwenSlerp2-14B 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
99 sometimesanotion/Qwenvergence-14B-v11 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
100 mergekit-community/mergekit-model_stock-skvbpno 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
101 sthenno/tempesthenno-ppo-ckpt40 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
102 maldv/Qwentile2.5-32B-Instruct 32.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
103 sthenno-com/miscii-14b-1225 14.8B 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
104 Qwen2.5-14B-Instruct-1M 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
105 deepcogito/cogito-v1-preview-qwen-14B 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
106 aixonlab/Zara-14b-v1.2 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
107 Nohobby/Q2.5-Qwetiapin-32B 32.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
108 mergekit-community/mergekit-model_stock-zelysxr 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
109 jpacifico/Chocolatine-2-14B-Instruct-v2.0b3 14.8B 0.64 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
110 jpacifico/Chocolatine-2-14B-Instruct-v2.0.3 14.8B 0.64 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
111 JungZoona/T3Q-qwen2.5-14b-v1.0-e3 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
112 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V3-70B 70.6B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
113 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V2-70B 70.6B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
114 sometimesanotion/Qwenvergence-14B-v10 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
115 fluently-lm/FluentlyLM-Prinum 32.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
116 prithivMLmods/Alpha-UMa-Qwen-14B 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
117 Sao10K/14B-Qwen2.5-Kunou-v1 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
118 sometimesanotion/Qwen2.5-14B-Vimarckoso-v3 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
119 Qwen2.5-14B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
120 arcee-ai/Virtuoso-Small 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
121 soob3123/amoral-cogito-14b 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
122 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V4-70B 70.6B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
123 arcee-ai/Virtuoso-Small-v2 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
124 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V1-32B 32.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
125 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V1-70B 70.6B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
126 rubenroy/Zurich-14B-GCv2-5m 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
127 mergekit-community/mergekit-model_stock-grrwjhl 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
128 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V8-70B 70.6B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
129 Ttimofeyka/Tissint-14B-v1.2-128k-RP 14.8B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
130 Cogito v1 Preview 70B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2025 huggingface.co Official leaderboard
131 Saxo/Linkbricks-Horizon-AI-Avengers-V6-32B 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
132 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V7-70B 70.6B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
133 nicoboss/DeepSeek-R1-Distill-Qwen-32B-Uncensored 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
134 Saxo/Linkbricks-Horizon-AI-Avengers-V3-32B 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
135 SicariusSicariiStuff/Impish_QWEN_14B-1M 14.8B 0.63 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
136 Sakalti/oxyge1-33B 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
137 Sakalti/Oxyge1-33B 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
138 hotmailuser/Deeepseek-QwenSlerp4-32B 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
139 TeamDelta/Re-ultiima-14B 14.8B 0.63 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
140 YOYO-AI/Qwen2.5-14B-YOYO-Average 14.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
141 Saxo/Linkbricks-Horizon-AI-Avengers-V1-32B 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
142 CultriX/Qwen2.5-14B-HyperSeekv5 14.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
143 Qwen/Qwen2.5-14B-Instruct 14.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
144 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V3-70B 70.6B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
145 Sakalti/ultiima-14B 14.8B 0.63 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
146 Llama 3-70B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
147 netease-youdao/Confucius-o1-14B 14.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
148 tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4 70.6B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
149 nitky/EZO-QwQ-32B-Preview 32.8B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
150 meta-llama/Meta-Llama-3-70B-Instruct 70.6B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
151 DataPilot/SKYDRIVE-32B-v0.1 32.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
152 tanliboy/lambda-qwen2.5-14b-dpo-test 14.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
153 deep-analysis-research/test-qwen2.5-14b-wo-system-v1 14.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
154 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V6-70B 70.6B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
155 AXCXEPT/EZO-Qwen2.5-32B-Instruct 32.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
156 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-70B 70.6B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
157 nitky/Llama-3.3-FakeSwallow-70B-Instruct-v0.1 70.6B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
158 Saxo/Linkbricks-Horizon-AI-Avengers-V2-32B 32.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
159 tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.3 70.6B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
160 tokyotech-llm/Llama-3.1-Swallow-70B-Instruct-v0.1 70.6B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
161 OpenBuddy/openbuddy-qwq-32b-v24.1-200k 32.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
162 Qwen2.5-Coder-32B-Instruct 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Nov 2024 huggingface.co Official leaderboard
163 DataPilot/Arrival-32B-Instruct-v0.5 32.8B 0.62 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
164 nitky/Llama-3.3-SuperSwallow-70B-Instruct-v0.1 70.6B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
165 rombodawg/Rombos-Coder-V2.5-Qwen-32b 32.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
166 wanlige/li-14b-v0.4-slerp0.1 14.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
167 nbeerbower/DeepSeek-R1-Qwen-lorablated-32B 32.8B 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
168 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V5-70B 70.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
169 Llama 3.3 70B Instruct 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
170 Qwen/Qwen2-57B-A14B 57.4B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
171 shisa-ai/shisa-v2-mistral-small-24b 23.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
172 nitky/RoguePlanet-DeepSeek-R1-Qwen-32B 32.8B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
173 Saxo/Linkbricks-Horizon-AI-Avengers-V4-32B 32.8B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
174 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V8-70B 70.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
175 Saxo/Linkbricks-Horizon-AI-Japanese-Avengers-V2-70B 70.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
176 Saxo/Linkbricks-Horizon-AI-Llama-3.3-Japanese-70B-sft-dpo-base 70.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
177 Saxo/Linkbricks-Horizon-AI-Avengers-V5-32B 32.8B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
178 Steelskull/L3.3-MS-Nevoria-70b 70.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
179 RDson/WomboCombo-R1-Coder-14B-Preview 14.8B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
180 Saxo/Linkbricks-Horizon-AI-Japanese-Base-70B 70.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
181 Qwen2.5-32B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
182 shisa-ai/shisa-v2-llama3.3-70b 70.6B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
183 wanlige/li-14b-v0.4 14.8B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
184 Ba2han/QwQenSeek-coder 32.8B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
185 TeetouchQQ/model_mergev2 32.8B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
186 Azure99/Blossom-V6-32B 32.8B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
187 abeja/ABEJA-Qwen2.5-32b-Japanese-v1.0 32.8B 0.60 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
188 cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese 32.8B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
189 abeja/ABEJA-Qwen2.5-32b-Japanese-v0.1 32.8B 0.60 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
190 FINGU-AI/FINGU-2.5-instruct-32B 32.8B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
191 FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview 32.8B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
192 DreadPoor/Irix-12B-Model_Stock 12.3B 0.60 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
193 RDson/CoderO1-DeepSeekR1-Coder-32B-Preview 32.8B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
194 spow12/ChatWaifu_v1.4 12.3B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
195 Llama 3.1-70B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jul 2024 huggingface.co Official leaderboard
196 Gen-Verse/ReasonFlux-F1 32.8B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
197 DeepSeek-R1-Distill-Qwen-32B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
198 Saxo/Linkbricks-Horizon-AI-Llama-3.3-Japanese-70B-Advanced 70.6B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
199 NaniDAO/deepseek-r1-qwen-2.5-32B-ablated 32.8B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
200 llm-jp/llm-jp-3.1-13b 13.7B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
201 prithivMLmods/Qwen2.5-14B-DeepSeek-R1-1M 14.8B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
202 spow12/ChatWaifu_12B_v2.0 12.3B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
203 QwQ-32B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Mar 2025 huggingface.co Official leaderboard
204 Saxo/Linkbricks-Horizon-AI-Korean-Superb-22B 22.3B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
205 Elizezen/Himeyuri-v0.1-12B 12.3B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
206 DeepSeek-R1-Distill-Llama-70B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
207 mistralai/Mistral-Nemo-Instruct-2407 12.3B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
208 google/gemma-2-27b-it 27.2B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
209 anthracite-org/magnum-v2.5-12b-kto 12.3B 0.58 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
210 Groq/Llama-3-Groq-70B-Tool-Use 70.6B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
211 karakuri-ai/karakuri-lm-8x7b-chat-v0.1 46.7B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
212 llm-jp/llm-jp-3.1-1.8b-instruct4 1.9B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
213 Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B 27.2B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
214 Azure99/Blossom-V6-14B 14.8B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
215 huihui-ai/DeepSeek-R1-Distill-Qwen-32B-abliterated 32.8B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
216 shisa-ai/shisa-v2-mistral-nemo-12b 12.3B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
217 spow12/ChatWaifu_v2.0_22B 22.3B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
218 CohereForAI/aya-23-35B 35B 0.57 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
219 Zhihu-ai/Zhi-writing-dsr1-14b 14.8B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
220 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-22B 22.3B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
221 agentica-org/DeepCoder-14B-Preview 14.8B 0.57 4-shot / float32 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
222 nvidia/AceReason-Nemotron-14B 14.8B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
223 DeepSeek-R1-Distill-Qwen-14B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
224 qingy2024/NaturalLM 12.3B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
225 qihoo360/TinyR1-32B-Preview 32.8B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
226 Qwen1.5-32B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
227 rubenroy/Geneva-12B-GCv2-5m 12.3B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
228 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.2 8B 0.56 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
229 cyberagent/DeepSeek-R1-Distill-Qwen-14B-Japanese 14.8B 0.56 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
230 Gen-Verse/ReasonFlux-F1-14B 14.8B 0.56 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
231 cyberagent/calm3-22b-chat 22.5B 0.56 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
232 Saxo/Linkbricks-Horizon-AI-Japanese-Superb-V1-27B 27.2B 0.56 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
233 elyza/Llama-3-ELYZA-JP-8B 8B 0.56 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
234 shisa-ai/shisa-v2-qwen2.5-7b 7.6B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
235 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.3 8B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
236 shisa-ai/shisa-v1-llama3-70b 70.6B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
237 tokyotech-llm/Llama-3.1-Swallow-8B-Instruct-v0.1 8B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
238 gmonsoon/SahabatAI-Lion-9B-TIES-v1 9.2B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
239 Qwen/Qwen2-57B-A14B-Instruct 57.4B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
240 tokyotech-llm/Llama-3.1-Swallow-8B-v0.1 8B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
241 Saxo/Linkbricks-Horizon-AI-Superb-27B 27.2B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
242 karakuri-ai/karakuri-lm-8x7b-instruct-v0.1 46.7B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
243 augmxnt/shisa-gamma-7b-v1 7.2B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
244 llm-jp/llm-jp-3-13b-instruct 13.7B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
245 Saxo/Linkbricks-Horizon-AI-Japanese-Advanced-V7-70B 70.6B 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
246 Reka Flash 3.1 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 May 2025 huggingface.co Official leaderboard
247 Qwen1.5-14B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
248 nbeerbower/Mistral-Nemo-Prism-12B-v2 12.3B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
249 rinna/llama-3-youko-8b-instruct 8B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
250 Kendamarron/Qwen2.5-7B-o1-ja-v0.1 7.6B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
251 lightblue/qwen2.5-7B-instruct-simpo 7.6B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
252 cyberagent/Mistral-Nemo-Japanese-Instruct-2408 12.3B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
253 meta-llama/Llama-2-70b-hf 69B 0.54 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
254 Qwen/Qwen2.5-7B-Instruct-1M 7.6B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
255 AIDC-AI/Marco-o1 7.6B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
256 Qwen2.5-7B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
257 tokyotech-llm/Llama-3.1-Swallow-8B-v0.2 8B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
258 Qwen2.5-Coder-14B-Instruct 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Nov 2024 huggingface.co Official leaderboard
259 rombodawg/Rombos-Coder-V2.5-Qwen-14b 14.8B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
260 Sakalti/deneb-v1-7b 7.6B 0.54 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
261 Sakalti/Saka-7.6B 7.6B 0.54 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
262 Saxo/Linkbricks-Horizon-AI-Korean-Advanced-56B 46.7B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
263 suayptalha/HomerCreativeAnvita-Mix-Qw7B 7.6B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
264 Qwen/Qwen1.5-32B-Chat 32.5B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
265 shisa-ai/shisa-v2-llama3.1-8b 8B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
266 DataPilot/Llama3.1-ArrowSE-v0.4 8B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
267 Azure99/Blossom-V6-7B 7.6B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
268 Nurture-intelligence/kEy-llama3.1-8b-v0.1 8B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
269 meta-llama/Llama-3.1-8B-Instruct 8B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
270 tokyo-electron-device-ai/llama3-tedllm-8b-v0-annealing 8.1B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
271 qihoo360/Light-R1-14B-DS 14.8B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
272 tokyotech-llm/Gemma-2-Llama-Swallow-27b-it-v0.1 27.2B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
273 Qwen/Qwen2.5-7B-Instruct 7.6B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
274 Qwen2-VL-7B-Instruct 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Aug 2024 huggingface.co Official leaderboard
275 AXCXEPT/EZO-Humanities-9B-gemma-2-it 9.2B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
276 fblgit/pancho-v1-qw25-3B-UNAMGS 3.4B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
277 AXCXEPT/EZO-Common-9B-gemma-2-it 9.2B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
278 deepcogito/cogito-v1-preview-llama-8B 8B 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
279 Sakalti/sakalinear-1-1.0 7.6B 0.53 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
280 AIDC-AI/Marco-LLM-GLO 7.6B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
281 rubenroy/Zurich-7B-GCv2-5m 0.52 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
282 Manual-Dataset-Creation-Project/Matsu-7B 7.6B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
283 Qwen2.5-3B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
284 Qwen2-7B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
285 google/gemma-2-9b-it 9.2B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
286 MaziyarPanahi/calme-3.2-instruct-3b 3.1B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
287 karakuri-ai/karakuri-lm-70b-chat-v0.1 69.2B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
288 Elizezen/Kudryavka-8B-alpha 8B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
289 Qwen2-7B-Instruct 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
290 Aya Expanse 8B 0.52 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
291 huihui-ai/DeepSeek-R1-Distill-Qwen-14B-abliterated-v2 14.8B 0.51 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
292 Sakalti/Neptuno-Alpha 3.4B 0.51 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
293 microsoft/Phi-3.5-mini-instruct 3.8B 0.51 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
294 Aya Expanse 32B 0.51 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
295 Casual-Autopsy/Llama-3-VNTL-Yollisa-8B 8B 0.51 4-shot / float32 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
296 huihui-ai/aya-expanse-32b-abliterated 32.3B 0.51 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
297 Sakalti/Lunar-4B 3.8B 0.51 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
298 ryota39/Tora-12B 12.3B 0.50 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
299 SakanaAI/EvoLLM-JP-A-v1-7B 7.2B 0.50 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
300 Qwen1.5-14B-Chat 0.50 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
301 meta-llama/Meta-Llama-3-8B-Instruct 8B 0.50 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
302 llm-jp/llm-jp-3.1-1.8b 1.9B 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
303 Granite-3.1-8B-Instruct 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
304 Llama 3.1-8B 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jul 2024 huggingface.co Official leaderboard
305 Granite 3.2 8B Instruct 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2025 huggingface.co Official leaderboard
306 Llama 3-8B 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
307 llm-jp/llm-jp-3-13b 13.7B 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
308 Qwen/Qwen2.5-Coder-7B-Instruct 7.6B 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
309 CohereForAI/aya-23-8B 8B 0.48 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
310 Qwen/Qwen2.5-3B-Instruct 3.1B 0.48 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
311 meta-llama/Llama-2-70b-chat-hf 69B 0.48 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
312 Qwen1.5-7B 0.47 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
313 tiiuae/Falcon3-10B-base 10.3B 0.47 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
314 Rakuten/RakutenAI-7B-chat 7.4B 0.46 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
315 deepcogito/cogito-v1-preview-llama-3B 3.6B 0.46 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
316 llm-jp/llm-jp-3-3.7b-instruct 3.8B 0.46 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
317 augmxnt/shisa-7b-v1 8B 0.46 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
318 Granite-3.1-8B-Base 0.46 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
319 Elizezen/Phos-7B 7.2B 0.46 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
320 Qwen2.5-1.5B 0.45 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
321 Qwen1.5-7B-Chat 0.45 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
322 Sakalti/Saba1.5-Pro 1.5B 0.45 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
323 Sakalti/Saba2-Preview 1.5B 0.45 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
324 Sakalti/tara-3.8B 3.8B 0.45 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
325 Sakalti/Tara-3.8B-v1.1 3.8B 0.45 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
326 jpacifico/Chocolatine-3B-Instruct-DPO-Revised 3.8B 0.45 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
327 cognitivecomputations/Dolphin3.0-R1-Mistral-24B 23.6B 0.45 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
328 Rakuten/RakutenAI-2.0-mini-instruct 1.5B 0.45 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
329 rinna/gemma-2-baku-2b-it 2.6B 0.45 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
330 Sakalti/Saka-1.5B 1.8B 0.45 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
331 Qwen/Qwen2.5-1.5B-Instruct 1.5B 0.44 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
332 Falcon3-10B-Instruct 0.44 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
333 Sakalti/SJT-1.5B-Alpha 1.8B 0.44 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
334 Qwen/Qwen1.5-MoE-A2.7B 14.3B 0.44 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
335 mobiuslabsgmbh/DeepSeek-R1-ReDistill-Llama3-8B-v1.1 8B 0.43 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
336 Granite-3.1-2B-Instruct 0.43 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
337 TIGER-Lab/Qwen2.5-Math-7B-CFT 7.6B 0.43 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
338 llm-jp/llm-jp-3-3.7b-instruct3 3.8B 0.43 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
339 llm-jp/llm-jp-3-3.7b-instruct2 3.8B 0.43 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
340 Saxo/Linkbricks-Horizon-AI-Korean-LLAMA3blend-8x8b 47.5B 0.43 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
341 meta-llama/Llama-2-13b-hf 13B 0.43 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
342 Llama 3.2 3B 0.42 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
343 tiiuae/Falcon3-7B-base 7.5B 0.42 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
344 deepseek-ai/DeepSeek-R1-Distill-Llama-8B 8B 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
345 meta-llama/Llama-3.2-3B-Instruct 3.2B 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
346 Saxo/Linkbricks-Horizon-AI-Japanese-Pro-V6-70B 70.6B 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
347 Granite-3.1-2B-Base 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
348 Qwen2-1.5B-Instruct 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
349 meta-llama/Llama-2-13b-chat-hf 13B 0.41 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
350 Gemma 2 2B IT 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jul 2024 huggingface.co Official leaderboard
351 Qwen/Qwen1.5-MoE-A2.7B-Chat 14.3B 0.40 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
352 Elizezen/SniffyOtter-7B 7.2B 0.40 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
353 Elizezen/Antler-7B 7.2B 0.40 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
354 rubenroy/Zurich-1.5B-GCv2-5m 1.5B 0.40 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
355 Qwen1.5-4B-Chat 0.40 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
356 DeepSeek-R1-Distill-Qwen-7B 0.39 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
357 llm-jp/llm-jp-3-1.8b-instruct 1.9B 0.39 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
358 Qwen/Qwen1.5-4B 4B 0.39 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
359 Qwen2-1.5B 0.39 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
360 mobiuslabsgmbh/DeepSeek-R1-ReDistill-Qwen-7B-v1.1 7.6B 0.39 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
361 Gen-Verse/ReasonFlux-F1-7B 7.6B 0.38 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
362 meta-llama/Llama-2-7b-chat-hf 6.7B 0.38 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
363 Granite-3.1-3B-A800M-Instruct 0.38 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
364 Sakalti/SJT-2.4B-Alpha 2.2B 0.38 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
365 weblab-GENIAC/Tanuki-8B-dpo-v1.0 7.5B 0.38 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
366 llm-jp/llm-jp-3-3.7b 3.8B 0.38 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
367 Saxo/Linkbricks-Horizon-AI-Superb-70B 70.6B 0.38 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
368 meta-llama/Llama-2-7b-hf 6.7B 0.37 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
369 Granite-3.1-3B-A800M-Base 0.37 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
370 Falcon3-7B-Instruct 0.35 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
371 cyberagent/calm2-7b-chat 7B 0.35 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
372 Elizezen/Hameln-japanese-mistral-7B 7.2B 0.34 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
373 llm-jp/llm-jp-3-1.8b 1.9B 0.34 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
374 prithivMLmods/Qwen2.5-32B-DeepSeek-R1-Instruct 32.8B 0.33 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
375 llm-jp/llm-jp-3-980m-instruct2 1B 0.33 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
376 llm-jp/llm-jp-3-980m-instruct3 1B 0.33 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
377 Qwen/Qwen2.5-Math-7B-Instruct 7.6B 0.32 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
378 Qwen2.5-0.5B 0.31 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
379 SmolLM2 1.7B Instruct 0.31 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
380 Qwen/Qwen2.5-0.5B-Instruct 0.5B 0.31 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
381 Sakalti/CodeSaka-7.6B 7.6B 0.31 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
382 meta-llama/Llama-3.2-1B-Instruct 1.2B 0.31 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
383 tiiuae/Falcon3-3B-base 3.2B 0.30 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
384 Qwen/Qwen1.5-1.8B 1.8B 0.30 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
385 Qwen/Qwen1.5-1.8B-Chat 1.8B 0.30 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
386 Qwen2-0.5B 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
387 Qwen2-0.5B-Instruct 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
388 Granite-3.1-1B-A400M-Instruct 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
389 llm-jp/llm-jp-3-980m 1B 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
390 yamatazen/Twilight-SCE-12B-v2 12.3B 0.29 0-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
391 Gemma 2B 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
392 Gemma 1.1 2B IT 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
393 sail/Sailor2-1B-Chat 1B 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
394 Falcon3-3B-Instruct 0.28 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
395 Llama 3.2 1B 0.28 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
396 google/gemma-2b-it 2.5B 0.27 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
397 Granite-3.1-1B-A400M-Base 0.27 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
398 agentica-org/DeepScaleR-1.5B-Preview 1.8B 0.26 4-shot / float32 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
399 agentica-org/DeepCoder-1.5B-Preview 1.8B 0.26 4-shot / float32 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
400 DeepSeek-R1-Distill-Qwen-1.5B 0.26 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
401 llm-jp/llm-jp-3-440m-instruct2 0.4B 0.25 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
402 llm-jp/llm-jp-3-440m-instruct3 0.4B 0.25 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
403 Falcon3-1B-Instruct 0.24 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
404 llm-jp/llm-jp-3-440m 0.4B 0.24 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
405 rsh345/llama3.1-8b-swallow-openmath-dare_ties-d5w5_d5w5 8B 0.24 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
406 nvidia/OpenMath2-Llama3.1-8B 8B 0.23 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
407 Qwen/Qwen1.5-0.5B 0.6B 0.23 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
408 tiiuae/Falcon3-1B-base 1.7B 0.21 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
409 Sakalti/MathSaka-7.6B 7.6B 0.21 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
410 CjangCjengh/GaLLM-multi-14B-v0.1 14.8B 0.21 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
411 CjangCjengh/GaLLM-14B-v0.2 14.8B 0.19 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
412 Qwen/Qwen1.5-0.5B-Chat 0.6B 0.19 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
413 llm-jp/llm-jp-3-150m 0.2B 0.18 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
414 HuggingFaceTB/SmolLM2-135M-Instruct 0.1B 0.16 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
415 google/gemma-7b-it 8.5B 0.15 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
416 Gemma 1.1 7B IT 0.15 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
417 Gemma 7B 0.04 0-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
418 Sakalti/Saka-7.2B 7.3B 0.03 0-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 huggingface.co Official leaderboard
#ModelScoreSettingDateLinkSource
1 QwQ-32B-Preview 0.67 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Nov 2024 huggingface.co Official leaderboard
2 Llama-3.1-Nemotron-70B-Instruct 0.66 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
3 Qwen2.5-14B-Instruct-1M 0.65 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
4 Qwen2.5-14B 0.64 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
5 Cogito v1 Preview 70B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2025 huggingface.co Official leaderboard
6 Llama 3-70B 0.63 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
7 Qwen2.5-Coder-32B-Instruct 0.62 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Nov 2024 huggingface.co Official leaderboard
8 Llama 3.3 70B Instruct 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
9 Qwen2.5-32B 0.61 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
10 Llama 3.1-70B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jul 2024 huggingface.co Official leaderboard
11 DeepSeek-R1-Distill-Qwen-32B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
12 QwQ-32B 0.59 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Mar 2025 huggingface.co Official leaderboard
13 DeepSeek-R1-Distill-Llama-70B 0.58 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
14 DeepSeek-R1-Distill-Qwen-14B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
15 Qwen1.5-32B 0.57 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
16 Reka Flash 3.1 0.55 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 May 2025 huggingface.co Official leaderboard
17 Qwen1.5-14B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
18 Qwen2.5-7B 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
19 Qwen2.5-Coder-14B-Instruct 0.54 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Nov 2024 huggingface.co Official leaderboard
20 Qwen2-VL-7B-Instruct 0.53 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Aug 2024 huggingface.co Official leaderboard
21 Qwen2.5-3B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
22 Qwen2-7B 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
23 Qwen2-7B-Instruct 0.52 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
24 Aya Expanse 8B 0.52 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
25 Aya Expanse 32B 0.51 4-shot / float16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
26 Qwen1.5-14B-Chat 0.50 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
27 Granite-3.1-8B-Instruct 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
28 Llama 3.1-8B 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jul 2024 huggingface.co Official leaderboard
29 Granite 3.2 8B Instruct 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2025 huggingface.co Official leaderboard
30 Llama 3-8B 0.49 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
31 Qwen1.5-7B 0.47 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
32 Granite-3.1-8B-Base 0.46 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
33 Qwen2.5-1.5B 0.45 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
34 Qwen1.5-7B-Chat 0.45 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
35 Falcon3-10B-Instruct 0.44 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
36 Granite-3.1-2B-Instruct 0.43 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
37 Llama 3.2 3B 0.42 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
38 Granite-3.1-2B-Base 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
39 Qwen2-1.5B-Instruct 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
40 Gemma 2 2B IT 0.41 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jul 2024 huggingface.co Official leaderboard
41 Qwen1.5-4B-Chat 0.40 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
42 DeepSeek-R1-Distill-Qwen-7B 0.39 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
43 Qwen2-1.5B 0.39 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
44 Granite-3.1-3B-A800M-Instruct 0.38 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
45 Granite-3.1-3B-A800M-Base 0.37 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
46 Falcon3-7B-Instruct 0.35 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
47 Qwen2.5-0.5B 0.31 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
48 SmolLM2 1.7B Instruct 0.31 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Oct 2024 huggingface.co Official leaderboard
49 Qwen2-0.5B 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
50 Qwen2-0.5B-Instruct 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jun 2024 huggingface.co Official leaderboard
51 Granite-3.1-1B-A400M-Instruct 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
52 Gemma 2B 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard
53 Gemma 1.1 2B IT 0.29 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
54 Falcon3-3B-Instruct 0.28 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
55 Llama 3.2 1B 0.28 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Sep 2024 huggingface.co Official leaderboard
56 Granite-3.1-1B-A400M-Base 0.27 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
57 DeepSeek-R1-Distill-Qwen-1.5B 0.26 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Jan 2025 huggingface.co Official leaderboard
58 Falcon3-1B-Instruct 0.24 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Dec 2024 huggingface.co Official leaderboard
59 Gemma 1.1 7B IT 0.15 4-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Apr 2024 huggingface.co Official leaderboard
60 Gemma 7B 0.04 0-shot / bfloat16 / main / vLLM v0.6.3.post1, llm-jp-eval v1.4.1 Feb 2024 huggingface.co Official leaderboard

Rankings by part

We read 54 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as AVG (NLI), AVG (QA) and AVG (RC). Aya Expanse 32B leads 17 of them, and 12 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.

AVG (NLI)

60 models we track have a score here.

  1. 1 Cogito v1 Preview 70B ↑4 0.83
  2. 2 Llama 3.3 70B Instruct ↑6 0.80
  3. 3 QwQ-32B-Preview ↓2 0.79
  4. 4 Llama-3.1-Nemotron-70B-Instruct ↓2 0.79
  5. 5 Qwen2.5-32B ↑4 0.78
  6. 6 Qwen2.5-14B-Instruct-1M ↓3 0.78
  7. 7 DeepSeek-R1-Distill-Qwen-32B ↑4 0.77
  8. 8 DeepSeek-R1-Distill-Qwen-14B ↑6 0.76
  9. 9 Qwen2.5-14B ↓5 0.76
  10. 10 DeepSeek-R1-Distill-Llama-70B ↑3 0.75

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
QwQ-32B-Preview llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.67
Llama-3.1-Nemotron-70B-Instruct llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.66
Qwen2.5-14B-Instruct-1M llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.65
Qwen2.5-14B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.64
Cogito v1 Preview 70B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.63
Llama 3-70B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.63
Qwen2.5-Coder-32B-Instruct llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.62
Llama 3.3 70B Instruct llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.61
Qwen2.5-32B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.61
Llama 3.1-70B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.59
DeepSeek-R1-Distill-Qwen-32B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.59
QwQ-32B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.59

Source: Official Open Japanese LLM Leaderboard leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
QwQ-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.67
Llama-3.1-Nemotron-70B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.66
Qwen2.5-14B-Instruct-1M 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.65
Qwen2.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.64
Cogito v1 Preview 70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.63
Llama 3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.63
Qwen2.5-Coder-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.62
Llama 3.3 70B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.61
Qwen2.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.61
Llama 3.1-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.59
DeepSeek-R1-Distill-Qwen-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.59
QwQ-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.59
DeepSeek-R1-Distill-Llama-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.58
QwQ-32B-Preview 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.58
DeepSeek-R1-Distill-Qwen-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.57
Qwen1.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.57
Reka Flash 3.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.55
Qwen1.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.54
Qwen2.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.54
Qwen2.5-Coder-14B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.54
Qwen2.5-14B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.53
Qwen2-VL-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.53
Qwen2.5-3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Qwen2-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Qwen2-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Aya Expanse 8B 4-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Llama-3.1-Nemotron-70B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.51
Aya Expanse 32B 4-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.51
Llama 3-70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Cogito v1 Preview 70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Qwen2.5-14B-Instruct-1M 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Qwen1.5-14B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Granite-3.1-8B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Llama 3.1-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Granite 3.2 8B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Llama 3-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Aya Expanse 32B 0-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.48
Qwen2.5-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.47
Qwen1.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.47
Qwen2.5-Coder-32B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.46
Granite-3.1-8B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.46
Qwen2.5-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.45
Qwen1.5-7B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.45
Falcon3-10B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.44
Granite-3.1-2B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.43
Llama 3.2 3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.42
Llama 3.1-70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.42
Granite-3.1-2B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Qwen2-1.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Qwen1.5-14B-Chat 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Gemma 2 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Qwen1.5-4B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.40
Qwen1.5-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
DeepSeek-R1-Distill-Qwen-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
DeepSeek-R1-Distill-Qwen-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
Qwen2-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
Granite 3.2 8B Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
Granite-3.1-3B-A800M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Reka Flash 3.1 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Qwen1.5-14B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Granite-3.1-8B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Llama 3.3 70B Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Qwen2-7B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Aya Expanse 8B 0-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Granite-3.1-3B-A800M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Qwen2-VL-7B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.36
Falcon3-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.35
Qwen2.5-3B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.34
Qwen2.5-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.34
Llama 3.1-8B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.33
QwQ-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.33
Qwen2-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.33
Qwen2.5-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.31
SmolLM2 1.7B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.31
Qwen2-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Llama 3-8B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Qwen2-0.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Granite-3.1-1B-A400M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Gemma 2B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Gemma 1.1 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Qwen1.5-7B-Chat 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Falcon3-7B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
DeepSeek-R1-Distill-Qwen-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Falcon3-3B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Llama 3.2 1B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Granite-3.1-2B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.27
Granite-3.1-1B-A400M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.27
Qwen1.5-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.26
DeepSeek-R1-Distill-Qwen-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.26
Qwen2.5-1.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.25
Qwen2-1.5B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.25
Falcon3-1B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.24
Granite-3.1-3B-A800M-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.23
DeepSeek-R1-Distill-Llama-70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.23
Granite-3.1-8B-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.22
Qwen2-1.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.22
DeepSeek-R1-Distill-Qwen-14B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.22
Llama 3.2 3B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.21
Qwen2.5-Coder-14B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.20
Granite-3.1-1B-A400M-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Gemma 2 2B IT 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Granite-3.1-2B-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Qwen2.5-0.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Qwen2-0.5B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.17
Falcon3-3B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.16
Falcon3-10B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
DeepSeek-R1-Distill-Qwen-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
Gemma 1.1 7B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
Granite-3.1-3B-A800M-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
DeepSeek-R1-Distill-Qwen-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.14
Granite-3.1-1B-A400M-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.13
Qwen2-0.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.12
SmolLM2 1.7B Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.11
Llama 3.2 1B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.11
Qwen1.5-4B-Chat 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.10
Gemma 1.1 2B IT 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.10
Gemma 2B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.07
Falcon3-1B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.07
Gemma 1.1 7B IT 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.07
DeepSeek-R1-Distill-Qwen-1.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.06
Gemma 7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.04
Gemma 7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.04

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
QwQ-32B-Preview llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.67
Llama-3.1-Nemotron-70B-Instruct llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.66
Qwen2.5-14B-Instruct-1M llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.65
Qwen2.5-14B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.64
Cogito v1 Preview 70B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.63
Llama 3-70B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.63
Qwen2.5-Coder-32B-Instruct llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.62
Llama 3.3 70B Instruct llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.61
Qwen2.5-32B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.61
Llama 3.1-70B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.59
DeepSeek-R1-Distill-Qwen-32B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.59
QwQ-32B llm-jp-eval v1.4.1
0-shot / bf… 4-shot / bf…
0.59

Source: Official Open Japanese LLM Leaderboard leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
QwQ-32B-Preview 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.67
Llama-3.1-Nemotron-70B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.66
Qwen2.5-14B-Instruct-1M 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.65
Qwen2.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.64
Cogito v1 Preview 70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.63
Llama 3-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.63
Qwen2.5-Coder-32B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.62
Llama 3.3 70B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.61
Qwen2.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.61
Llama 3.1-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.59
DeepSeek-R1-Distill-Qwen-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.59
QwQ-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.59
DeepSeek-R1-Distill-Llama-70B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.58
QwQ-32B-Preview 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.58
DeepSeek-R1-Distill-Qwen-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.57
Qwen1.5-32B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.57
Reka Flash 3.1 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.55
Qwen1.5-14B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.54
Qwen2.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.54
Qwen2.5-Coder-14B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.54
Qwen2.5-14B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.53
Qwen2-VL-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.53
Qwen2.5-3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Qwen2-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Qwen2-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Aya Expanse 8B 4-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.52
Llama-3.1-Nemotron-70B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.51
Aya Expanse 32B 4-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.51
Llama 3-70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Cogito v1 Preview 70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Qwen2.5-14B-Instruct-1M 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Qwen1.5-14B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.50
Granite-3.1-8B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Llama 3.1-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Granite 3.2 8B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Llama 3-8B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.49
Aya Expanse 32B 0-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.48
Qwen2.5-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.47
Qwen1.5-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.47
Qwen2.5-Coder-32B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.46
Granite-3.1-8B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.46
Qwen2.5-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.45
Qwen1.5-7B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.45
Falcon3-10B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.44
Granite-3.1-2B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.43
Llama 3.2 3B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.42
Llama 3.1-70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.42
Granite-3.1-2B-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Qwen2-1.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Qwen1.5-14B-Chat 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Gemma 2 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.41
Qwen1.5-4B-Chat 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.40
Qwen1.5-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
DeepSeek-R1-Distill-Qwen-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
DeepSeek-R1-Distill-Qwen-7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
Qwen2-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
Granite 3.2 8B Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.39
Granite-3.1-3B-A800M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Reka Flash 3.1 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Qwen1.5-14B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Granite-3.1-8B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.38
Llama 3.3 70B Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Qwen2-7B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Aya Expanse 8B 0-shot / float16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Granite-3.1-3B-A800M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.37
Qwen2-VL-7B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.36
Falcon3-7B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.35
Qwen2.5-3B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.34
Qwen2.5-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.34
Llama 3.1-8B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.33
QwQ-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.33
Qwen2-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.33
Qwen2.5-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.31
SmolLM2 1.7B Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.31
Qwen2-0.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Llama 3-8B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Qwen2-0.5B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Granite-3.1-1B-A400M-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Gemma 2B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Gemma 1.1 2B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.29
Qwen1.5-7B-Chat 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Falcon3-7B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
DeepSeek-R1-Distill-Qwen-32B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Falcon3-3B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Llama 3.2 1B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.28
Granite-3.1-2B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.27
Granite-3.1-1B-A400M-Base 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.27
Qwen1.5-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.26
DeepSeek-R1-Distill-Qwen-1.5B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.26
Qwen2.5-1.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.25
Qwen2-1.5B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.25
Falcon3-1B-Instruct 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.24
Granite-3.1-3B-A800M-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.23
DeepSeek-R1-Distill-Llama-70B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.23
Granite-3.1-8B-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.22
Qwen2-1.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.22
DeepSeek-R1-Distill-Qwen-14B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.22
Llama 3.2 3B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.21
Qwen2.5-Coder-14B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.20
Granite-3.1-1B-A400M-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Gemma 2 2B IT 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Granite-3.1-2B-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Qwen2.5-0.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.18
Qwen2-0.5B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.17
Falcon3-3B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.16
Falcon3-10B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
DeepSeek-R1-Distill-Qwen-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
Gemma 1.1 7B IT 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
Granite-3.1-3B-A800M-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.15
DeepSeek-R1-Distill-Qwen-7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.14
Granite-3.1-1B-A400M-Base 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.13
Qwen2-0.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.12
SmolLM2 1.7B Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.11
Llama 3.2 1B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.11
Qwen1.5-4B-Chat 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.10
Gemma 1.1 2B IT 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.10
Gemma 2B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.07
Falcon3-1B-Instruct 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.07
Gemma 1.1 7B IT 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.07
DeepSeek-R1-Distill-Qwen-1.5B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.06
Gemma 7B 0-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.04
Gemma 7B 4-shot / bfloat16 / main / vLLM v0.6.3.post1 llm-jp-eval v1.4.1 0.04

2 models have held number one on the official leaderboard since its release in November 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

NVIDIA Qwen
Own leaderboard Not tested since May 2025

Source: Official Open Japanese LLM Leaderboard leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 0.20 0.40 0.60 0.80 2024 2025 0.00 0.20 0.40 0.60 0.80 2024 2025

Source: Official Open Japanese LLM Leaderboard leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Multilingual

All 545 in Multilingual
  • MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
  • WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET. K-EXAONE 90.5%
  • MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language. Qwen3.7-Max 87%
  • CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories. Solar Open 2 90.7%
  • Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
  • MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%

Data audit

8 of 9 checks pass, 1 warning. Look first at the Newest score check. The newest came 18 months ago, in May 2025, for Reka Flash 3.1. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 1 warning 2 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (418 entries).
  • Pass Models scored 60 models we track, plus 358 other entries on its leaderboard.
  • Warning Newest score The newest came 18 months ago, in May 2025, for Reka Flash 3.1.
  • Note Frontier models None of the top 10 has a score. The strongest tested is QwQ-32B (#137).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on huggingface.co: 122 rows for models we track and 358 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check