Benchmark Almanac

ImageJevBench v0.3.0

kettei-fat (Sqwish) 75.4 tops its leaderboard GPT-6 Luna 17.3 leads the models we track
Not in model cards
Version 0.3.0

Models make decisions about images and report probabilities. This version draws 300 public and 1,200 sealed items from a new pool.

Benchmark Heaven’s core results list 68 measured systems and accept 65 for ranking. The maker keeps older scores and computer use separate.

Released
October 2026
Kind
General visual questions
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
Public, plus a private test set
Measures
Capability
Language
English
In and out
Text, Images → Text
Interaction
Single turn
Run by
A host
In views
Vision
Strongest model tested
Gemini 3.8 Flash #24 on the capability index
Last model added
September 2026
Scores refreshed
November 20th, 2026
64
Entries on its own leaderboard, 4 models we track

kettei-fat (Sqwish) tops the leaderboard with 75.4 GPT-6 Luna leads with 17.3 among models released by September 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is GPT-6 Luna, at 17.3. This view shows only models we track, each at its best.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

64 entries. Scroll sideways for the rest.

  1. 75.4 kettei-fat (Sqwish)
  2. 71.2 Wity-1 (AlphaNimble), reasoning auto
  3. 66.6 H2O-Lightning-4B v1.4.1 (image)
  4. 63.7 Imajev-4B
  5. 63.3 kettei-chub (Sqwish)
  6. 51.2 H2O-Lightning-31B v1.0.7 (image)
  7. 51.0 H2O-Lightning-31B v1.3.4 @c35ed950 (image)
  8. 47.9 Sage 1.4 + images, beta (Levanto Labs)
  9. 44.6 Drex Max (Nace AI)
  10. 42.5 Surogate Rune 26B-A4B v3
  11. 41.4 Autoloops – Gemma 4 31B IT
  12. 40.9 basal-omni-1.0-ultra (Remek Kinas)
  13. 39.5 Celeris-1 Decision (Celeris)
  14. 34.5 imajev 9B
  15. 34.1 H2O-Lightning-4B (image)
  16. 33.1 Wity-1 (AlphaNimble), reasoning off
  17. 32.8 Jev-Omni
  18. 31.4 JPT-9B (kirp / llm2jev)
  19. 30.4 NeoHorse Jev 4B
  20. 29.7 Jevify Gemma 4 26B-A4B
  21. 29.3 AutoJev-27B
  22. 26.6 JPT-4B (kirp / llm2jev)
  23. 26.4 Jev 1.13.0 + images, preview (TypeSafe)
  24. 25.5 Jev-Vision 8B (SeanLiu)
  25. 24.3 Jev 1.14.0 + images, preview (TypeSafe)
  26. 22.8 shisa-de-1
  27. 20.4 Clef-Flash (Cloudflare)
  28. 18.6 Clef (Cloudflare)
  29. 17.3 GPT-6 Luna
  30. 17.1 Winnow-12B (Q8_0 + F16 vision projector)
  31. 15.3 Image Hopper (HopitAI)
  32. 15.0 OmniJev-Qwen3.5-9B-v4 (tzcfly)
  33. 14.8 OmniJev 4B (tinnel123)
  34. 13.8 Reflex 4B (released stable configuration)
  35. 12.0 Bonsai-2-27B v2 PQ2_0 + Q8_0 MMProj
  36. 11.5 CUA-S1 4B
  37. 11.1 Visual-Jev generic (zero-shot Qwen3.5-4B baseline)
  38. 9.52 Mapika decider-2b-vision BF16
  39. 8.86 vjev-vision (yah01)
  40. 8.71 Visual-Jev 4B Answer-SFT
  41. 7.87 GPT-5.6 Luna
  42. 7.26 Jevify Gemma 4 E4B
  43. 7.17 JPT-0.8B (kirp / llm2jev)
  44. 6.85 Glance (frozen Qwen3-VL-4B)
  45. 6.45 Gemini 3.1 Flash-Lite
  46. 5.90 diffusiongemma-26b djev v10 step160 (snowicarus)
  47. 5.76 imajev 2B
  48. 4.14 Gevva E4B (text checkpoint, image input)
  49. 4.10 Liquid d1-3B (Liquid AI)
  50. 3.62 djev-distill-v4 (tarsur385)
  51. 3.29 Jevify Qwen3-VL-2B T2
  52. 3.17 OmniJev 2B (tinnel123)
  53. 2.02 Gevva E2B multimodal
  54. 1.97 jev-spatial (Fr0zencr4nE)
  55. 1.84 JEVision (divyanshx11, visual route)
  56. 1.57 OmniJev 0.8B (tinnel123)
  57. 0.50 PlayJev 0.8B
  58. 0.24 Liquid d1-omni-600M (Liquid AI)
  59. 0.04 djev-dev BF16
  60. 0.03 Qevi-2B (MeerDevelopment)
  61. 0.00 djev-spark NVFP4
  62. 0.00 Gemini 3.8 Flash
  63. 0.00 OpenJev 4B NLI v2 (official image-premise path)
  64. 0.00 OpenJev 4B NLI v5
  1. 1 kettei-fat (Sqwish) 75.4
  2. 2 Wity-1 (AlphaNimble), reasoning auto auto 71.2
  3. 3 H2O-Lightning-4B v1.4.1 (image) 66.6
  4. 4 Imajev-4B 63.7
  5. 5 kettei-chub (Sqwish) 63.3
  6. 6 H2O-Lightning-31B v1.0.7 (image) 51.2
  7. 7 H2O-Lightning-31B v1.3.4 @c35ed950 (image) 51.0
  8. 8 Sage 1.4 + images, beta (Levanto Labs) 47.9
  9. 9 Drex Max (Nace AI) 44.6
  10. 10 Surogate Rune 26B-A4B v3 42.5
  11. 11 Autoloops – Gemma 4 31B IT 41.4
  12. 12 basal-omni-1.0-ultra (Remek Kinas) 40.9
  13. 13 Celeris-1 Decision (Celeris) 39.5
  14. 14 imajev 9B 34.5
  15. 15 H2O-Lightning-4B (image) 34.1
  16. 16 Wity-1 (AlphaNimble), reasoning off off 33.1
  17. 17 Jev-Omni 32.8
  18. 18 JPT-9B (kirp / llm2jev) 31.4
  19. 19 NeoHorse Jev 4B 30.4
  20. 20 Jevify Gemma 4 26B-A4B 29.7
  21. 21 AutoJev-27B 29.3
  22. 22 JPT-4B (kirp / llm2jev) 26.6
  23. 23 Jev 1.13.0 + images, preview (TypeSafe) 26.4
  24. 24 Jev-Vision 8B (SeanLiu) 25.5
  25. 25 Jev 1.14.0 + images, preview (TypeSafe) 24.3
  26. 26 shisa-de-1 22.8
  27. 27 Clef-Flash (Cloudflare) 20.4
  28. 28 Clef (Cloudflare) 18.6
  29. 29 GPT-6 Luna 17.3
  30. 30 Winnow-12B (Q8_0 + F16 vision projector) 17.1
  31. 31 Image Hopper (HopitAI) 15.3
  32. 32 OmniJev-Qwen3.5-9B-v4 (tzcfly) 15.0
  33. 33 OmniJev 4B (tinnel123) 14.8
  34. 34 Reflex 4B (released stable configuration) 13.8
  35. 35 Bonsai-2-27B v2 PQ2_0 + Q8_0 MMProj 12.0
  36. 36 CUA-S1 4B 11.5
  37. 37 Visual-Jev generic (zero-shot Qwen3.5-4B baseline) 11.1
  38. 38 Mapika decider-2b-vision BF16 9.52
  39. 39 vjev-vision (yah01) 8.86
  40. 40 Visual-Jev 4B Answer-SFT 8.71
  41. 41 GPT-5.6 Luna 7.87
  42. 42 Jevify Gemma 4 E4B 7.26
  43. 43 JPT-0.8B (kirp / llm2jev) 7.17
  44. 44 Glance (frozen Qwen3-VL-4B) 6.85
  45. 45 Gemini 3.1 Flash-Lite 6.45
  46. 46 diffusiongemma-26b djev v10 step160 (snowicarus) 5.90
  47. 47 imajev 2B 5.76
  48. 48 Gevva E4B (text checkpoint, image input) 4.14
  49. 49 Liquid d1-3B (Liquid AI) 4.10
  50. 50 djev-distill-v4 (tarsur385) 3.62
  51. 51 Jevify Qwen3-VL-2B T2 3.29
  52. 52 OmniJev 2B (tinnel123) 3.17
  53. 53 Gevva E2B multimodal 2.02
  54. 54 jev-spatial (Fr0zencr4nE) 1.97
  55. 55 JEVision (divyanshx11, visual route) 1.84
  56. 56 OmniJev 0.8B (tinnel123) 1.57
  57. 57 PlayJev 0.8B 0.50
  58. 58 Liquid d1-omni-600M (Liquid AI) 0.24
  59. 59 djev-dev BF16 0.04
  60. 60 Qevi-2B (MeerDevelopment) 0.03
  61. 61 djev-spark NVFP4 0.00
  62. 62 Gemini 3.8 Flash 0.00
  63. 63 OpenJev 4B NLI v2 (official image-premise path) 0.00
  64. 64 OpenJev 4B NLI v5 0.00

Source: Official ImageJevBench v0.3.0 leaderboard

#EntryScoreSettingReleasedRun onLinkSource
1 kettei-fat (Sqwish) closed weights 75.4 Oct 2026 docs.sqwish.ai Official leaderboard
2 Wity-1 (AlphaNimble), reasoning auto closed weights 71.2 auto Oct 2026 wity.alphanimble.com Official leaderboard
3 H2O-Lightning-4B v1.4.1 (image) 66.6 Oct 2026 huggingface.co Official leaderboard
4 Imajev-4B 63.7 Oct 2026 Official leaderboard
5 kettei-chub (Sqwish) closed weights 63.3 Oct 2026 docs.sqwish.ai Official leaderboard
6 H2O-Lightning-31B v1.0.7 (image) 51.2 Oct 2026 huggingface.co Official leaderboard
7 H2O-Lightning-31B v1.3.4 @c35ed950 (image) 51.0 Nov 2026 huggingface.co Official leaderboard
8 Sage 1.4 + images, beta (Levanto Labs) closed weights 47.9 Oct 2026 docs.levanto.ai Official leaderboard
9 Drex Max (Nace AI) closed weights 44.6 Oct 2026 drex.nace.ai Official leaderboard
10 Surogate Rune 26B-A4B v3 42.5 Oct 2026 huggingface.co Official leaderboard
11 Autoloops – Gemma 4 31B IT closed weights 41.4 Oct 2026 Official leaderboard
12 basal-omni-1.0-ultra (Remek Kinas) 40.9 Oct 2026 huggingface.co Official leaderboard
13 Celeris-1 Decision (Celeris) closed weights 39.5 Oct 2026 docs.celeris.ai Official leaderboard
14 imajev 9B 34.5 Oct 2026 huggingface.co Official leaderboard
15 H2O-Lightning-4B (image) 34.1 Oct 2026 huggingface.co Official leaderboard
16 Wity-1 (AlphaNimble), reasoning off closed weights 33.1 off Oct 2026 wity.alphanimble.com Official leaderboard
17 Jev-Omni 32.8 Oct 2026 Official leaderboard
18 JPT-9B (kirp / llm2jev) 31.4 Oct 2026 huggingface.co Official leaderboard
19 NeoHorse Jev 4B 30.4 Oct 2026 huggingface.co Official leaderboard
20 Jevify Gemma 4 26B-A4B 29.7 Oct 2026 huggingface.co Official leaderboard
21 AutoJev-27B 29.3 Oct 2026 huggingface.co Official leaderboard
22 JPT-4B (kirp / llm2jev) 26.6 Oct 2026 huggingface.co Official leaderboard
23 Jev 1.13.0 + images, preview (TypeSafe) closed weights 26.4 Oct 2026 docs.typesafe.ai Official leaderboard
24 Jev-Vision 8B (SeanLiu) 25.5 Oct 2026 huggingface.co Official leaderboard
25 Jev 1.14.0 + images, preview (TypeSafe) closed weights 24.3 Oct 2026 docs.typesafe.ai Official leaderboard
26 shisa-de-1 22.8 Oct 2026 huggingface.co Official leaderboard
27 Clef-Flash (Cloudflare) 20.4 Oct 2026 Official leaderboard
28 Clef (Cloudflare) 18.6 Oct 2026 Official leaderboard
29 GPT-6 Luna 17.3 OpenAI Decisions Sep 2026 Oct 2026 Official leaderboard
30 Winnow-12B (Q8_0 + F16 vision projector) 17.1 Oct 2026 Official leaderboard
31 Image Hopper (HopitAI) 15.3 Oct 2026 Official leaderboard
32 OmniJev-Qwen3.5-9B-v4 (tzcfly) 15.0 Oct 2026 huggingface.co Official leaderboard
33 OmniJev 4B (tinnel123) 14.8 Oct 2026 huggingface.co Official leaderboard
34 Reflex 4B (released stable configuration) 13.8 Oct 2026 Official leaderboard
35 Bonsai-2-27B v2 PQ2_0 + Q8_0 MMProj 12.0 Oct 2026 Official leaderboard
36 CUA-S1 4B 11.5 Oct 2026 huggingface.co Official leaderboard
37 Visual-Jev generic (zero-shot Qwen3.5-4B baseline) 11.1 Oct 2026 huggingface.co Official leaderboard
38 Mapika decider-2b-vision BF16 9.52 Oct 2026 Official leaderboard
39 vjev-vision (yah01) 8.86 Oct 2026 huggingface.co Official leaderboard
40 Visual-Jev 4B Answer-SFT 8.71 Oct 2026 huggingface.co Official leaderboard
41 GPT-5.6 Luna 7.87 Jul 2026 Oct 2026 Official leaderboard
42 Jevify Gemma 4 E4B 7.26 Oct 2026 huggingface.co Official leaderboard
43 JPT-0.8B (kirp / llm2jev) 7.17 Oct 2026 huggingface.co Official leaderboard
44 Glance (frozen Qwen3-VL-4B) 6.85 Oct 2026 huggingface.co Official leaderboard
45 Gemini 3.1 Flash-Lite 6.45 Mar 2026 Oct 2026 Official leaderboard
46 diffusiongemma-26b djev v10 step160 (snowicarus) 5.90 Oct 2026 huggingface.co Official leaderboard
47 imajev 2B 5.76 Oct 2026 huggingface.co Official leaderboard
48 Gevva E4B (text checkpoint, image input) 4.14 Oct 2026 huggingface.co Official leaderboard
49 Liquid d1-3B (Liquid AI) 4.10 Oct 2026 huggingface.co Official leaderboard
50 djev-distill-v4 (tarsur385) 3.62 Oct 2026 huggingface.co Official leaderboard
51 Jevify Qwen3-VL-2B T2 3.29 Oct 2026 huggingface.co Official leaderboard
52 OmniJev 2B (tinnel123) 3.17 Oct 2026 huggingface.co Official leaderboard
53 Gevva E2B multimodal 2.02 Oct 2026 huggingface.co Official leaderboard
54 jev-spatial (Fr0zencr4nE) 1.97 Oct 2026 huggingface.co Official leaderboard
55 JEVision (divyanshx11, visual route) 1.84 Oct 2026 huggingface.co Official leaderboard
56 OmniJev 0.8B (tinnel123) 1.57 Oct 2026 huggingface.co Official leaderboard
57 PlayJev 0.8B 0.50 Oct 2026 huggingface.co Official leaderboard
58 Liquid d1-omni-600M (Liquid AI) 0.24 Oct 2026 huggingface.co Official leaderboard
59 djev-dev BF16 0.04 Oct 2026 Official leaderboard
60 Qevi-2B (MeerDevelopment) 0.03 Oct 2026 huggingface.co Official leaderboard
61 djev-spark NVFP4 0.00 Oct 2026 Official leaderboard
62 Gemini 3.8 Flash 0.00 Sep 2026 Oct 2026 Official leaderboard
63 OpenJev 4B NLI v2 (official image-premise path) 0.00 Oct 2026 Official leaderboard
64 OpenJev 4B NLI v5 0.00 Oct 2026 huggingface.co Official leaderboard
#ModelScoreSettingReleasedRun onLinkSource
1 GPT-6 Luna 17.3 OpenAI Decisions Sep 2026 Oct 2026 Official leaderboard
2 GPT-5.6 Luna 7.87 Jul 2026 Oct 2026 Official leaderboard
3 Gemini 3.1 Flash-Lite 6.45 Mar 2026 Oct 2026 Official leaderboard
4 Gemini 3.8 Flash 0.00 Sep 2026 Oct 2026 Official leaderboard

Rankings by part

We read 2 parts from the benchmark’s own leaderboard: Public accuracy and Sealed accuracy. Each ranks models on one part of the test. GPT-5.6 Luna leads both. Arrows show how far a model moved from its overall rank among the models we track.

Public accuracy

4 models we track have a score here.

  1. 1 GPT-5.6 Luna ↑1 86.4%
  2. 2 GPT-6 Luna ↓1 83.4%
  3. 3 Gemini 3.1 Flash-Lite 58.7%
  4. 4 Gemini 3.8 Flash 36.6%

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Oct 2026
GPT-6 Luna 17.3

Source: Official ImageJevBench v0.3.0 leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 5.00 10.0 15.0 20.0 2026 0.00 5.00 10.0 15.0 20.0 2026

Source: Official ImageJevBench v0.3.0 leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

Versions

More in Vision

All 636 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. Qwen 3.6 Plus 86%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy. Seed2.1 Pro 90.7%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access Public tasks, plus a private test set.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (64 entries).
  • Pass Models scored 4 models we track, plus 60 other entries on its leaderboard.
  • Pass Newest score The newest is from September 2026, for GPT-6 Luna.
  • Note Frontier models None of the top 10 has a score. The strongest tested is Gemini 3.8 Flash (#24).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on benchmarkheaven.com: 5 rows for models we track and 60 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check