Benchmark Almanac

CIRR

mongodb/voyage-multimodal-3 35% tops its leaderboard EmbeddingGemma 2 22% leads the models we track
Not in model cards

Models find an image that matches a reference photograph after applying a change described in text.

The target is another real photograph.

Made by
Zheyuan Liu and colleagues
Released
August 2021
Kind
General visual questions
Level
Everyday
Scored
Checked automatically
Answers
Prediction
Tasks
All public
Measures
Capability
Language
English
In and out
Text, Images → Text
Interaction
Single turn
Items from
Real work and use
Refresh
One fixed set
Run by
Anyone
In views
Vision
Last model added
October 2026
Scores refreshed
November 20th, 2026
52
Entries on MTEB’s leaderboard, 1 model we track

mongodb/voyage-multimodal-3 tops the leaderboard with 35% EmbeddingGemma 2 leads with 22%

Best score per entry from MTEB’s leaderboard, read on November 20th, 2026. MTEB reports test-set nDCG at 10, shown here as a percentage. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is EmbeddingGemma 2, at 22%. This view shows only models we track, each at its best.

52 entries. Scroll sideways for the rest.

  1. 35.0 mongodb/voyage-multimodal-3
  2. 29.0 QuanSun/EVA02-CLIP-bigE-14-plus
  3. 29.0 LCO-Embedding/LCO-Embedding-Omni-7B
  4. 28.3 royokong/e5-v
  5. 27.6 QuanSun/EVA02-CLIP-bigE-14
  6. 27.5 LCO-Embedding/LCO-Embedding-Omni-3B
  7. 23.6 QuanSun/EVA02-CLIP-L-14
  8. 22.3 Haon-Chen/e5-omni-7B
  9. 22.0 EmbeddingGemma 2
  10. 21.7 Haon-Chen/e5-omni-3B
  11. 21.2 kakaobrain/align-base
  12. 21.2 QuanSun/EVA02-CLIP-B-16
  13. 20.3 google/siglip-so400m-patch14-384
  14. 19.9 Salesforce/blip-itm-large-flickr
  15. 19.4 Salesforce/blip-itm-large-coco
  16. 18.6 dnotitia/DNA-VL-STEER-2B
  17. 17.2 Salesforce/blip-itm-base-coco
  18. 16.5 Salesforce/blip-itm-base-flickr
  19. 16.5 BidirLM/BidirLM-Omni-2.5B-Embedding
  20. 15.9 Tevatron/OmniEmbed-v0.1
  21. 15.8 jinaai/jina-clip-v1
  22. 15.8 google/siglip-large-patch16-384
  23. 15.7 TIGER-Lab/VLM2Vec-Full
  24. 15.7 TIGER-Lab/VLM2Vec-LoRA
  25. 15.5 google/siglip-base-patch16-384
  26. 15.5 BAAI/bge-visualized-m3
  27. 15.4 BAAI/bge-visualized-base
  28. 15.4 google/siglip-base-patch16-224
  29. 15.4 google/siglip-base-patch16-512
  30. 15.3 google/siglip-large-patch16-256
  31. 15.3 google/siglip-base-patch16-256
  32. 15.0 encord-team/ebind-full
  33. 13.8 laion/CLIP-ViT-bigG-14-laion2B-39B-b160k
  34. 13.7 LanguageBind/LanguageBind_Omni
  35. 13.1 google/siglip-base-patch16-256-multilingual
  36. 12.4 laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K
  37. 12.3 laion/CLIP-ViT-H-14-laion2B-s32B-b79K
  38. 12.2 laion/CLIP-ViT-g-14-laion2B-s34B-b88K
  39. 12.2 laion/CLIP-ViT-L-14-laion2B-s32B-b82K
  40. 11.9 laion/CLIP-ViT-B-32-laion2B-s34B-b79K
  41. 11.9 laion/CLIP-ViT-B-16-DataComp.XL-s13B-b90K
  42. 11.9 laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K
  43. 9.92 openai/clip-vit-base-patch32
  44. 9.53 openai/clip-vit-base-patch16
  45. 9.51 nomic-ai/nomic-embed-vision-v1.5
  46. 9.40 openai/clip-vit-large-patch14
  47. 8.07 jinaai/jina-embeddings-v5-omni-small
  48. 7.57 jinaai/jina-embeddings-v5-omni-nano
  49. 7.36 Salesforce/blip2-opt-2.7b
  50. 7.35 Salesforce/blip2-opt-6.7b-coco
  51. 6.57 Salesforce/blip-image-captioning-large
  52. 5.70 Salesforce/blip-image-captioning-base
  1. 1 mongodb/voyage-multimodal-3 35%
  2. 2 QuanSun/EVA02-CLIP-bigE-14-plus 29%
  3. 3 LCO-Embedding/LCO-Embedding-Omni-7B 29%
  4. 4 royokong/e5-v 28.3%
  5. 5 QuanSun/EVA02-CLIP-bigE-14 27.6%
  6. 6 LCO-Embedding/LCO-Embedding-Omni-3B 27.5%
  7. 7 QuanSun/EVA02-CLIP-L-14 23.6%
  8. 8 Haon-Chen/e5-omni-7B 22.3%
  9. 9 EmbeddingGemma 2 22%
  10. 10 Haon-Chen/e5-omni-3B 21.7%
  11. 11 kakaobrain/align-base 21.2%
  12. 12 QuanSun/EVA02-CLIP-B-16 21.2%
  13. 13 google/siglip-so400m-patch14-384 20.3%
  14. 14 Salesforce/blip-itm-large-flickr 19.9%
  15. 15 Salesforce/blip-itm-large-coco 19.4%
  16. 16 dnotitia/DNA-VL-STEER-2B 18.6%
  17. 17 Salesforce/blip-itm-base-coco 17.2%
  18. 18 Salesforce/blip-itm-base-flickr 16.5%
  19. 19 BidirLM/BidirLM-Omni-2.5B-Embedding 16.5%
  20. 20 Tevatron/OmniEmbed-v0.1 15.9%
  21. 21 jinaai/jina-clip-v1 15.8%
  22. 22 google/siglip-large-patch16-384 15.8%
  23. 23 TIGER-Lab/VLM2Vec-Full 15.7%
  24. 24 TIGER-Lab/VLM2Vec-LoRA 15.7%
  25. 25 google/siglip-base-patch16-384 15.5%
  26. 26 BAAI/bge-visualized-m3 15.5%
  27. 27 BAAI/bge-visualized-base 15.4%
  28. 28 google/siglip-base-patch16-224 15.4%
  29. 29 google/siglip-base-patch16-512 15.4%
  30. 30 google/siglip-large-patch16-256 15.3%
  31. 31 google/siglip-base-patch16-256 15.3%
  32. 32 encord-team/ebind-full 15%
  33. 33 laion/CLIP-ViT-bigG-14-laion2B-39B-b160k 13.8%
  34. 34 LanguageBind/LanguageBind_Omni 13.7%
  35. 35 google/siglip-base-patch16-256-multilingual 13.1%
  36. 36 laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K 12.4%
  37. 37 laion/CLIP-ViT-H-14-laion2B-s32B-b79K 12.3%
  38. 38 laion/CLIP-ViT-g-14-laion2B-s34B-b88K 12.2%
  39. 39 laion/CLIP-ViT-L-14-laion2B-s32B-b82K 12.2%
  40. 40 laion/CLIP-ViT-B-32-laion2B-s34B-b79K 11.9%
  41. 41 laion/CLIP-ViT-B-16-DataComp.XL-s13B-b90K 11.9%
  42. 42 laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K 11.9%
  43. 43 openai/clip-vit-base-patch32 9.92%
  44. 44 openai/clip-vit-base-patch16 9.53%
  45. 45 nomic-ai/nomic-embed-vision-v1.5 9.51%
  46. 46 openai/clip-vit-large-patch14 9.40%
  47. 47 jinaai/jina-embeddings-v5-omni-small 8.07%
  48. 48 jinaai/jina-embeddings-v5-omni-nano 7.57%
  49. 49 Salesforce/blip2-opt-2.7b 7.36%
  50. 50 Salesforce/blip2-opt-6.7b-coco 7.35%
  51. 51 Salesforce/blip-image-captioning-large 6.57%
  52. 52 Salesforce/blip-image-captioning-base 5.70%

Source: MTEB’s leaderboard

Source: MTEB’s leaderboard

#EntryScoreSettingDateLinkSource
1 mongodb/voyage-multimodal-3 closed weights 35% huggingface.co Official leaderboard
2 QuanSun/EVA02-CLIP-bigE-14-plus open weights, 5B 29% huggingface.co Official leaderboard
3 LCO-Embedding/LCO-Embedding-Omni-7B open weights, 8.9B 29% huggingface.co Official leaderboard
4 royokong/e5-v open weights, 8.4B 28.3% huggingface.co Official leaderboard
5 QuanSun/EVA02-CLIP-bigE-14 open weights, 4.7B 27.6% huggingface.co Official leaderboard
6 LCO-Embedding/LCO-Embedding-Omni-3B open weights, 4.7B 27.5% huggingface.co Official leaderboard
7 QuanSun/EVA02-CLIP-L-14 open weights, 0.4B 23.6% huggingface.co Official leaderboard
8 Haon-Chen/e5-omni-7B open weights, 8.9B 22.3% huggingface.co Official leaderboard
9 EmbeddingGemma 2 22% Oct 2026 huggingface.co Official leaderboard
10 Haon-Chen/e5-omni-3B open weights, 4.7B 21.7% huggingface.co Official leaderboard
11 kakaobrain/align-base open weights, 0.2B 21.2% huggingface.co Official leaderboard
12 QuanSun/EVA02-CLIP-B-16 open weights, 0.1B 21.2% huggingface.co Official leaderboard
13 google/siglip-so400m-patch14-384 open weights, 0.9B 20.3% huggingface.co Official leaderboard
14 Salesforce/blip-itm-large-flickr open weights, 0.4B 19.9% huggingface.co Official leaderboard
15 Salesforce/blip-itm-large-coco open weights, 0.4B 19.4% huggingface.co Official leaderboard
16 dnotitia/DNA-VL-STEER-2B open weights, 2.1B 18.6% huggingface.co Official leaderboard
17 Salesforce/blip-itm-base-coco open weights, 0.2B 17.2% huggingface.co Official leaderboard
18 Salesforce/blip-itm-base-flickr open weights, 0.2B 16.5% huggingface.co Official leaderboard
19 BidirLM/BidirLM-Omni-2.5B-Embedding open weights, 2.4B 16.5% huggingface.co Official leaderboard
20 Tevatron/OmniEmbed-v0.1 open weights, 8.9B 15.9% huggingface.co Official leaderboard
21 jinaai/jina-clip-v1 open weights, 0.2B 15.8% huggingface.co Official leaderboard
22 google/siglip-large-patch16-384 open weights, 0.7B 15.8% huggingface.co Official leaderboard
23 TIGER-Lab/VLM2Vec-Full open weights, 4.1B 15.7% huggingface.co Official leaderboard
24 TIGER-Lab/VLM2Vec-LoRA open weights, 4.2B 15.7% huggingface.co Official leaderboard
25 google/siglip-base-patch16-384 open weights, 0.2B 15.5% huggingface.co Official leaderboard
26 BAAI/bge-visualized-m3 open weights, 0.9B 15.5% huggingface.co Official leaderboard
27 BAAI/bge-visualized-base open weights, 0.2B 15.4% huggingface.co Official leaderboard
28 google/siglip-base-patch16-224 open weights, 0.2B 15.4% huggingface.co Official leaderboard
29 google/siglip-base-patch16-512 open weights, 0.2B 15.4% huggingface.co Official leaderboard
30 google/siglip-large-patch16-256 open weights, 0.7B 15.3% huggingface.co Official leaderboard
31 google/siglip-base-patch16-256 open weights, 0.2B 15.3% huggingface.co Official leaderboard
32 encord-team/ebind-full open weights, 1.8B 15% huggingface.co Official leaderboard
33 laion/CLIP-ViT-bigG-14-laion2B-39B-b160k open weights, 2.5B 13.8% huggingface.co Official leaderboard
34 LanguageBind/LanguageBind_Omni open weights, 1.2B 13.7% github.com Official leaderboard
35 google/siglip-base-patch16-256-multilingual open weights, 0.4B 13.1% huggingface.co Official leaderboard
36 laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K open weights, 0.4B 12.4% huggingface.co Official leaderboard
37 laion/CLIP-ViT-H-14-laion2B-s32B-b79K open weights, 1B 12.3% huggingface.co Official leaderboard
38 laion/CLIP-ViT-g-14-laion2B-s34B-b88K open weights, 1.4B 12.2% huggingface.co Official leaderboard
39 laion/CLIP-ViT-L-14-laion2B-s32B-b82K open weights, 0.4B 12.2% huggingface.co Official leaderboard
40 laion/CLIP-ViT-B-32-laion2B-s34B-b79K open weights, 0.2B 11.9% huggingface.co Official leaderboard
41 laion/CLIP-ViT-B-16-DataComp.XL-s13B-b90K open weights, 0.2B 11.9% huggingface.co Official leaderboard
42 laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K open weights, 0.2B 11.9% huggingface.co Official leaderboard
43 openai/clip-vit-base-patch32 open weights, 0.2B 9.92% huggingface.co Official leaderboard
44 openai/clip-vit-base-patch16 open weights, 0.2B 9.53% huggingface.co Official leaderboard
45 nomic-ai/nomic-embed-vision-v1.5 open weights, 0.1B 9.51% huggingface.co Official leaderboard
46 openai/clip-vit-large-patch14 open weights, 0.4B 9.40% huggingface.co Official leaderboard
47 jinaai/jina-embeddings-v5-omni-small open weights, 1.6B 8.07% huggingface.co Official leaderboard
48 jinaai/jina-embeddings-v5-omni-nano open weights, 1B 7.57% huggingface.co Official leaderboard
49 Salesforce/blip2-opt-2.7b open weights, 3.7B 7.36% huggingface.co Official leaderboard
50 Salesforce/blip2-opt-6.7b-coco open weights, 7.8B 7.35% huggingface.co Official leaderboard
51 Salesforce/blip-image-captioning-large open weights, 0.4B 6.57% huggingface.co Official leaderboard
52 Salesforce/blip-image-captioning-base open weights, 0.2B 5.70% huggingface.co Official leaderboard
#ModelScoreSettingDateLinkSource
1 EmbeddingGemma 2 22% Oct 2026 huggingface.co Official leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On MTEB’s leaderboard
0% 20% 40% 60% 80% 100% Near the ceiling (85%) EmbeddingGemma 2 0% 20% 40% 60% 80% 100% Near the ceiling (85%) EmbeddingGemma 2

Source: MTEB’s leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Vision

All 636 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. Qwen 3.6 Plus 86%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy. Seed2.1 Pro 90.7%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%

Data audit

All 10 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
  • Pass Independent runs Scores from the official leaderboard (52 entries).
  • Pass Models scored 1 model we track, plus 51 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for EmbeddingGemma 2.
  • Note Frontier models None of its models are on the capabilities index.
  • Pass Headroom The best is 22%, under 85%, so it still separates models.
Leaderboard import 3 pass
  • Pass Importer We read the board on MTEB: 1 row for models we track and 51 other entries at the last read.
  • Pass Last read November 20th, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check