Benchmark Almanac

ImageJevBench v0.1.5

Wity-1 80.2 tops its leaderboard GPT-6 Luna 16.1 leads the models we track
Not in model cards
Version 0.1.5

Models answer decisions about images and assign probabilities to the answers. The maker ranks capability using accuracy and calibration within cost and latency limits.

Benchmark Heaven’s archived board compares 50 systems on 228 public and 456 sealed items.

Released
October 2026
Kind
General visual questions
Level
Everyday
Scored
Checked automatically
Answers
Multiple choice
Tasks
Public, plus a private test set
Measures
Capability
Language
English
In and out
Images, Text → Text
Interaction
Single turn
Run by
A host
In views
Vision
Strongest model tested
Gemini 3.8 Flash #24 on the capability index
Last model added
September 2026
Scores refreshed
November 20th, 2026
50
Entries on its own leaderboard, 4 models we track

Wity-1 tops the leaderboard with 80.2 GPT-6 Luna leads with 16.1 among models released by September 2026

Best score per entry from the benchmark's own leaderboard, read on November 20th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is GPT-6 Luna, at 16.1. This view shows only models we track, each at its best.

The newest model with a score here came out in September 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

50 entries. Scroll sideways for the rest.

  1. 1 Wity-1 Wity SystemOne, reasoning=auto 80.2
  2. 2 Imajev-4B PyTorch; one rotation; --fast; --merge-lora; calibration.json; server 8501f5c3; adapter c9e5f132. 76.4
  3. 3 Jev-Omni 73.1
  4. 4 NeoHorse Jev 4B 71.9
  5. 5 Visual-Jev 4B Answer-SFT 69.8
  6. 6 JPT-4B (kirp / llm2jev) 69.5
  7. 7 imajev 2B 68.7
  8. 8 Glance (frozen Qwen3-VL-4B) 66.9
  9. 9 AutoJev-27B 66.8
  10. 10 Mapika decider-2b-vision BF16 66.8
  11. 11 jev-spatial (Fr0zencr4nE) 65.9
  12. 12 JPT-9B (kirp / llm2jev) 65.8
  13. 13 OmniJev 4B (tinnel123) 65.7
  14. 14 Reflex 4B (released stable configuration) 65.4
  15. 15 OmniJev-Qwen3.5-9B-v4 (tzcfly) 65.1
  16. 16 Jevify Gemma 4 26B-A4B 65.0
  17. 17 Surogate Rune 26B-A4B v3 63.8
  18. 18 JevAny-27B SFT 63.8
  19. 19 JevAny-27B RLCR 63.5
  20. 20 Jev-Vision 8B (SeanLiu) 63.4
  21. 21 shisa-de-1 63.2
  22. 22 OmniJev 2B (tinnel123) 62.8
  23. 23 CUA-S1 4B 62.6
  24. 24 Standard One 8B 59.8
  25. 25 Jevify Qwen3-VL-2B T2 59.0
  26. 26 imajev 9B 58.3
  27. 27 Visual-Jev generic (zero-shot Qwen3.5-4B baseline) 57.7
  28. 28 djev-distill-v4 (tarsur385) 50.0
  29. 29 djev-spark NVFP4 49.4
  30. 30 diffusiongemma-26b djev v10 step160 (snowicarus) 48.8
  31. 31 vjev-vision (yah01) 47.7
  32. 32 Autoloops – Gemma 4 31B IT 47.1
  33. 33 djev-dev BF16 46.6
  34. 34 Winnow-12B (Q8_0 + F16 vision projector) 43.6
  35. 35 JPT-0.8B (kirp / llm2jev) 30.5
  36. 36 Jevify Gemma 4 E4B 29.8
  37. 37 Standard One 3B 29.0
  38. 38 Gevva E4B (text checkpoint, image input) 26.8
  39. 39 OmniJev 0.8B (tinnel123) 26.8
  40. 40 Gevva E2B multimodal 21.9
  41. 41 JEVision (divyanshx11, visual route) 21.5
  42. 42 Bonsai-2-27B v2 PQ2_0 + Q8_0 MMProj 19.3
  43. 43 Qevi-2B (MeerDevelopment) 17.4
  44. 44 GPT-6 Luna OpenRouter reasoning.effort=low 16.1
  45. 45 GPT-5.6 Luna 11.1
  46. 46 Gemini 3.1 Flash-Lite 9.58
  47. 47 PlayJev 0.8B 0.11
  48. 48 Gemini 3.8 Flash 0.00
  49. 49 OpenJev 4B NLI v2 (official image-premise path) 0.00
  50. 50 OpenJev 4B NLI v5 0.00

Source: Official ImageJevBench v0.1.5 leaderboard

#EntryScoreSettingDateSource
1 Wity-1 80.2 Wity SystemOne, reasoning=auto Official leaderboard
2 Imajev-4B 76.4 PyTorch; one rotation; --fast; --merge-lora; calibration.json; server 8501f5c3; adapter c9e5f132. Official leaderboard
3 Jev-Omni 73.1 Official leaderboard
4 NeoHorse Jev 4B 71.9 Official leaderboard
5 Visual-Jev 4B Answer-SFT 69.8 Official leaderboard
6 JPT-4B (kirp / llm2jev) 69.5 Official leaderboard
7 imajev 2B 68.7 Official leaderboard
8 Glance (frozen Qwen3-VL-4B) 66.9 Official leaderboard
9 AutoJev-27B 66.8 Official leaderboard
10 Mapika decider-2b-vision BF16 66.8 Official leaderboard
11 jev-spatial (Fr0zencr4nE) 65.9 Official leaderboard
12 JPT-9B (kirp / llm2jev) 65.8 Official leaderboard
13 OmniJev 4B (tinnel123) 65.7 Official leaderboard
14 Reflex 4B (released stable configuration) 65.4 Official leaderboard
15 OmniJev-Qwen3.5-9B-v4 (tzcfly) 65.1 Official leaderboard
16 Jevify Gemma 4 26B-A4B 65.0 Official leaderboard
17 Surogate Rune 26B-A4B v3 63.8 Official leaderboard
18 JevAny-27B SFT 63.8 Official leaderboard
19 JevAny-27B RLCR 63.5 Official leaderboard
20 Jev-Vision 8B (SeanLiu) 63.4 Official leaderboard
21 shisa-de-1 63.2 Official leaderboard
22 OmniJev 2B (tinnel123) 62.8 Official leaderboard
23 CUA-S1 4B 62.6 Official leaderboard
24 Standard One 8B 59.8 Official leaderboard
25 Jevify Qwen3-VL-2B T2 59.0 Official leaderboard
26 imajev 9B 58.3 Official leaderboard
27 Visual-Jev generic (zero-shot Qwen3.5-4B baseline) 57.7 Official leaderboard
28 djev-distill-v4 (tarsur385) 50.0 Official leaderboard
29 djev-spark NVFP4 49.4 Official leaderboard
30 diffusiongemma-26b djev v10 step160 (snowicarus) 48.8 Official leaderboard
31 vjev-vision (yah01) 47.7 Official leaderboard
32 Autoloops – Gemma 4 31B IT 47.1 Official leaderboard
33 djev-dev BF16 46.6 Official leaderboard
34 Winnow-12B (Q8_0 + F16 vision projector) 43.6 Official leaderboard
35 JPT-0.8B (kirp / llm2jev) 30.5 Official leaderboard
36 Jevify Gemma 4 E4B 29.8 Official leaderboard
37 Standard One 3B 29.0 Official leaderboard
38 Gevva E4B (text checkpoint, image input) 26.8 Official leaderboard
39 OmniJev 0.8B (tinnel123) 26.8 Official leaderboard
40 Gevva E2B multimodal 21.9 Official leaderboard
41 JEVision (divyanshx11, visual route) 21.5 Official leaderboard
42 Bonsai-2-27B v2 PQ2_0 + Q8_0 MMProj 19.3 Official leaderboard
43 Qevi-2B (MeerDevelopment) 17.4 Official leaderboard
44 GPT-6 Luna 16.1 OpenRouter reasoning.effort=low Sep 2026 Official leaderboard
45 GPT-5.6 Luna 11.1 Jul 2026 Official leaderboard
46 Gemini 3.1 Flash-Lite 9.58 Mar 2026 Official leaderboard
47 PlayJev 0.8B 0.11 Official leaderboard
48 Gemini 3.8 Flash 0.00 Sep 2026 Official leaderboard
49 OpenJev 4B NLI v2 (official image-premise path) 0.00 Official leaderboard
50 OpenJev 4B NLI v5 0.00 Official leaderboard
#ModelScoreSettingDateSource
1 GPT-6 Luna 16.1 OpenRouter reasoning.effort=low Sep 2026 Official leaderboard
2 GPT-5.6 Luna 11.1 Jul 2026 Official leaderboard
3 Gemini 3.1 Flash-Lite 9.58 Mar 2026 Official leaderboard
4 Gemini 3.8 Flash 0.00 Sep 2026 Official leaderboard

Rankings by part

We read 4 parts from the benchmark’s own leaderboard, each ranking models on one part of the test, such as Core, Everyday photo and Public accuracy. GPT-6 Luna leads 2 of them, and GPT-5.6 Luna leads the other 2. Arrows show how far a model moved from its overall rank among the models we track.

Core

4 models we track have a score here.

  1. 1 GPT-6 Luna 17.1
  2. 2 GPT-5.6 Luna 10.9
  3. 3 Gemini 3.1 Flash-Lite 9.06
  4. 4 Gemini 3.8 Flash 0.00

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Replaced by 0.3.0, Oct 2026
GPT-6 Luna 16.1

Source: Official ImageJevBench v0.1.5 leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Plot with other benchmarks
On the benchmark’s own leaderboard
0.00 5.00 10.0 15.0 20.0 2026 0.00 5.00 10.0 15.0 20.0 2026

Source: Official ImageJevBench v0.1.5 leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

Versions

More in Vision

All 636 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. Qwen 3.6 Plus 86%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy. Seed2.1 Pro 90.7%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access Public tasks, plus a private test set.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (50 entries).
  • Pass Models scored 4 models we track, plus 46 other entries on its leaderboard.
  • Pass Newest score The newest is from September 2026, for GPT-6 Luna.
  • Note Frontier models None of the top 10 has a score, which is fine for a retired test. The strongest tested is Gemini 3.8 Flash (#24).
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on benchmarkheaven.com: 4 rows for models we track and 46 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check