CIRR
Models find an image that matches a reference photograph after applying a change described in text.
The target is another real photograph.
- Made by
- Zheyuan Liu and colleagues
- Released
- August 2021
- Kind
- General visual questions
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Prediction
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Items from
- Real work and use
- Refresh
- One fixed set
- Run by
- Anyone
- In views
- Vision
- Last model added
- October 2026
- Scores refreshed
- November 20th, 2026
mongodb/voyage-multimodal-3 tops the leaderboard with 35% EmbeddingGemma 2 leads with 22%
Best score per entry from MTEB’s leaderboard, read on November 20th, 2026. MTEB reports test-set nDCG at 10, shown here as a percentage. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is EmbeddingGemma 2, at 22%. This view shows only models we track, each at its best.
- 35.0 mongodb/voyage-multimodal-3
- 29.0 QuanSun/EVA02-CLIP-bigE-14-plus
- 29.0 LCO-Embedding/LCO-Embedding-Omni-7B
- 28.3 royokong/e5-v
- 27.6 QuanSun/EVA02-CLIP-bigE-14
- 27.5 LCO-Embedding/LCO-Embedding-Omni-3B
- 23.6 QuanSun/EVA02-CLIP-L-14
- 22.3 Haon-Chen/e5-omni-7B
- 22.0
EmbeddingGemma 2
- 21.7 Haon-Chen/e5-omni-3B
- 21.2 kakaobrain/align-base
- 21.2 QuanSun/EVA02-CLIP-B-16
- 20.3 google/siglip-so400m-patch14-384
- 19.9 Salesforce/blip-itm-large-flickr
- 19.4 Salesforce/blip-itm-large-coco
- 18.6 dnotitia/DNA-VL-STEER-2B
- 17.2 Salesforce/blip-itm-base-coco
- 16.5 Salesforce/blip-itm-base-flickr
- 16.5 BidirLM/BidirLM-Omni-2.5B-Embedding
- 15.9 Tevatron/OmniEmbed-v0.1
- 15.8 jinaai/jina-clip-v1
- 15.8 google/siglip-large-patch16-384
- 15.7 TIGER-Lab/VLM2Vec-Full
- 15.7 TIGER-Lab/VLM2Vec-LoRA
- 15.5 google/siglip-base-patch16-384
- 15.5 BAAI/bge-visualized-m3
- 15.4 BAAI/bge-visualized-base
- 15.4 google/siglip-base-patch16-224
- 15.4 google/siglip-base-patch16-512
- 15.3 google/siglip-large-patch16-256
- 15.3 google/siglip-base-patch16-256
- 15.0 encord-team/ebind-full
- 13.8 laion/CLIP-ViT-bigG-14-laion2B-39B-b160k
- 13.7 LanguageBind/LanguageBind_Omni
- 13.1 google/siglip-base-patch16-256-multilingual
- 12.4 laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K
- 12.3 laion/CLIP-ViT-H-14-laion2B-s32B-b79K
- 12.2 laion/CLIP-ViT-g-14-laion2B-s34B-b88K
- 12.2 laion/CLIP-ViT-L-14-laion2B-s32B-b82K
- 11.9 laion/CLIP-ViT-B-32-laion2B-s34B-b79K
- 11.9 laion/CLIP-ViT-B-16-DataComp.XL-s13B-b90K
- 11.9 laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K
- 9.92 openai/clip-vit-base-patch32
- 9.53 openai/clip-vit-base-patch16
- 9.51 nomic-ai/nomic-embed-vision-v1.5
- 9.40 openai/clip-vit-large-patch14
- 8.07 jinaai/jina-embeddings-v5-omni-small
- 7.57 jinaai/jina-embeddings-v5-omni-nano
- 7.36 Salesforce/blip2-opt-2.7b
- 7.35 Salesforce/blip2-opt-6.7b-coco
- 6.57 Salesforce/blip-image-captioning-large
- 5.70 Salesforce/blip-image-captioning-base
- 1 mongodb/voyage-multimodal-3 35%
- 2 QuanSun/EVA02-CLIP-bigE-14-plus 29%
- 3 LCO-Embedding/LCO-Embedding-Omni-7B 29%
- 4 royokong/e5-v 28.3%
- 5 QuanSun/EVA02-CLIP-bigE-14 27.6%
- 6 LCO-Embedding/LCO-Embedding-Omni-3B 27.5%
- 7 QuanSun/EVA02-CLIP-L-14 23.6%
- 8 Haon-Chen/e5-omni-7B 22.3%
- 9
EmbeddingGemma 2 22%
- 10 Haon-Chen/e5-omni-3B 21.7%
- 11 kakaobrain/align-base 21.2%
- 12 QuanSun/EVA02-CLIP-B-16 21.2%
- 13 google/siglip-so400m-patch14-384 20.3%
- 14 Salesforce/blip-itm-large-flickr 19.9%
- 15 Salesforce/blip-itm-large-coco 19.4%
- 16 dnotitia/DNA-VL-STEER-2B 18.6%
- 17 Salesforce/blip-itm-base-coco 17.2%
- 18 Salesforce/blip-itm-base-flickr 16.5%
- 19 BidirLM/BidirLM-Omni-2.5B-Embedding 16.5%
- 20 Tevatron/OmniEmbed-v0.1 15.9%
- 21 jinaai/jina-clip-v1 15.8%
- 22 google/siglip-large-patch16-384 15.8%
- 23 TIGER-Lab/VLM2Vec-Full 15.7%
- 24 TIGER-Lab/VLM2Vec-LoRA 15.7%
- 25 google/siglip-base-patch16-384 15.5%
- 26 BAAI/bge-visualized-m3 15.5%
- 27 BAAI/bge-visualized-base 15.4%
- 28 google/siglip-base-patch16-224 15.4%
- 29 google/siglip-base-patch16-512 15.4%
- 30 google/siglip-large-patch16-256 15.3%
- 31 google/siglip-base-patch16-256 15.3%
- 32 encord-team/ebind-full 15%
- 33 laion/CLIP-ViT-bigG-14-laion2B-39B-b160k 13.8%
- 34 LanguageBind/LanguageBind_Omni 13.7%
- 35 google/siglip-base-patch16-256-multilingual 13.1%
- 36 laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K 12.4%
- 37 laion/CLIP-ViT-H-14-laion2B-s32B-b79K 12.3%
- 38 laion/CLIP-ViT-g-14-laion2B-s34B-b88K 12.2%
- 39 laion/CLIP-ViT-L-14-laion2B-s32B-b82K 12.2%
- 40 laion/CLIP-ViT-B-32-laion2B-s34B-b79K 11.9%
- 41 laion/CLIP-ViT-B-16-DataComp.XL-s13B-b90K 11.9%
- 42 laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K 11.9%
- 43 openai/clip-vit-base-patch32 9.92%
- 44 openai/clip-vit-base-patch16 9.53%
- 45 nomic-ai/nomic-embed-vision-v1.5 9.51%
- 46 openai/clip-vit-large-patch14 9.40%
- 47 jinaai/jina-embeddings-v5-omni-small 8.07%
- 48 jinaai/jina-embeddings-v5-omni-nano 7.57%
- 49 Salesforce/blip2-opt-2.7b 7.36%
- 50 Salesforce/blip2-opt-6.7b-coco 7.35%
- 51 Salesforce/blip-image-captioning-large 6.57%
- 52 Salesforce/blip-image-captioning-base 5.70%
Source: MTEB’s leaderboard
Source: MTEB’s leaderboard
| # | Entry | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | mongodb/voyage-multimodal-3 closed weights | 35% | huggingface.co | Official leaderboard | ||
| 2 | QuanSun/EVA02-CLIP-bigE-14-plus open weights, 5B | 29% | huggingface.co | Official leaderboard | ||
| 3 | LCO-Embedding/LCO-Embedding-Omni-7B open weights, 8.9B | 29% | huggingface.co | Official leaderboard | ||
| 4 | royokong/e5-v open weights, 8.4B | 28.3% | huggingface.co | Official leaderboard | ||
| 5 | QuanSun/EVA02-CLIP-bigE-14 open weights, 4.7B | 27.6% | huggingface.co | Official leaderboard | ||
| 6 | LCO-Embedding/LCO-Embedding-Omni-3B open weights, 4.7B | 27.5% | huggingface.co | Official leaderboard | ||
| 7 | QuanSun/EVA02-CLIP-L-14 open weights, 0.4B | 23.6% | huggingface.co | Official leaderboard | ||
| 8 | Haon-Chen/e5-omni-7B open weights, 8.9B | 22.3% | huggingface.co | Official leaderboard | ||
| 9 | | 22% | Oct 2026 | huggingface.co | Official leaderboard | |
| 10 | Haon-Chen/e5-omni-3B open weights, 4.7B | 21.7% | huggingface.co | Official leaderboard | ||
| 11 | kakaobrain/align-base open weights, 0.2B | 21.2% | huggingface.co | Official leaderboard | ||
| 12 | QuanSun/EVA02-CLIP-B-16 open weights, 0.1B | 21.2% | huggingface.co | Official leaderboard | ||
| 13 | google/siglip-so400m-patch14-384 open weights, 0.9B | 20.3% | huggingface.co | Official leaderboard | ||
| 14 | Salesforce/blip-itm-large-flickr open weights, 0.4B | 19.9% | huggingface.co | Official leaderboard | ||
| 15 | Salesforce/blip-itm-large-coco open weights, 0.4B | 19.4% | huggingface.co | Official leaderboard | ||
| 16 | dnotitia/DNA-VL-STEER-2B open weights, 2.1B | 18.6% | huggingface.co | Official leaderboard | ||
| 17 | Salesforce/blip-itm-base-coco open weights, 0.2B | 17.2% | huggingface.co | Official leaderboard | ||
| 18 | Salesforce/blip-itm-base-flickr open weights, 0.2B | 16.5% | huggingface.co | Official leaderboard | ||
| 19 | BidirLM/BidirLM-Omni-2.5B-Embedding open weights, 2.4B | 16.5% | huggingface.co | Official leaderboard | ||
| 20 | Tevatron/OmniEmbed-v0.1 open weights, 8.9B | 15.9% | huggingface.co | Official leaderboard | ||
| 21 | jinaai/jina-clip-v1 open weights, 0.2B | 15.8% | huggingface.co | Official leaderboard | ||
| 22 | google/siglip-large-patch16-384 open weights, 0.7B | 15.8% | huggingface.co | Official leaderboard | ||
| 23 | TIGER-Lab/VLM2Vec-Full open weights, 4.1B | 15.7% | huggingface.co | Official leaderboard | ||
| 24 | TIGER-Lab/VLM2Vec-LoRA open weights, 4.2B | 15.7% | huggingface.co | Official leaderboard | ||
| 25 | google/siglip-base-patch16-384 open weights, 0.2B | 15.5% | huggingface.co | Official leaderboard | ||
| 26 | BAAI/bge-visualized-m3 open weights, 0.9B | 15.5% | huggingface.co | Official leaderboard | ||
| 27 | BAAI/bge-visualized-base open weights, 0.2B | 15.4% | huggingface.co | Official leaderboard | ||
| 28 | google/siglip-base-patch16-224 open weights, 0.2B | 15.4% | huggingface.co | Official leaderboard | ||
| 29 | google/siglip-base-patch16-512 open weights, 0.2B | 15.4% | huggingface.co | Official leaderboard | ||
| 30 | google/siglip-large-patch16-256 open weights, 0.7B | 15.3% | huggingface.co | Official leaderboard | ||
| 31 | google/siglip-base-patch16-256 open weights, 0.2B | 15.3% | huggingface.co | Official leaderboard | ||
| 32 | encord-team/ebind-full open weights, 1.8B | 15% | huggingface.co | Official leaderboard | ||
| 33 | laion/CLIP-ViT-bigG-14-laion2B-39B-b160k open weights, 2.5B | 13.8% | huggingface.co | Official leaderboard | ||
| 34 | LanguageBind/LanguageBind_Omni open weights, 1.2B | 13.7% | github.com | Official leaderboard | ||
| 35 | google/siglip-base-patch16-256-multilingual open weights, 0.4B | 13.1% | huggingface.co | Official leaderboard | ||
| 36 | laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K open weights, 0.4B | 12.4% | huggingface.co | Official leaderboard | ||
| 37 | laion/CLIP-ViT-H-14-laion2B-s32B-b79K open weights, 1B | 12.3% | huggingface.co | Official leaderboard | ||
| 38 | laion/CLIP-ViT-g-14-laion2B-s34B-b88K open weights, 1.4B | 12.2% | huggingface.co | Official leaderboard | ||
| 39 | laion/CLIP-ViT-L-14-laion2B-s32B-b82K open weights, 0.4B | 12.2% | huggingface.co | Official leaderboard | ||
| 40 | laion/CLIP-ViT-B-32-laion2B-s34B-b79K open weights, 0.2B | 11.9% | huggingface.co | Official leaderboard | ||
| 41 | laion/CLIP-ViT-B-16-DataComp.XL-s13B-b90K open weights, 0.2B | 11.9% | huggingface.co | Official leaderboard | ||
| 42 | laion/CLIP-ViT-B-32-DataComp.XL-s13B-b90K open weights, 0.2B | 11.9% | huggingface.co | Official leaderboard | ||
| 43 | openai/clip-vit-base-patch32 open weights, 0.2B | 9.92% | huggingface.co | Official leaderboard | ||
| 44 | openai/clip-vit-base-patch16 open weights, 0.2B | 9.53% | huggingface.co | Official leaderboard | ||
| 45 | nomic-ai/nomic-embed-vision-v1.5 open weights, 0.1B | 9.51% | huggingface.co | Official leaderboard | ||
| 46 | openai/clip-vit-large-patch14 open weights, 0.4B | 9.40% | huggingface.co | Official leaderboard | ||
| 47 | jinaai/jina-embeddings-v5-omni-small open weights, 1.6B | 8.07% | huggingface.co | Official leaderboard | ||
| 48 | jinaai/jina-embeddings-v5-omni-nano open weights, 1B | 7.57% | huggingface.co | Official leaderboard | ||
| 49 | Salesforce/blip2-opt-2.7b open weights, 3.7B | 7.36% | huggingface.co | Official leaderboard | ||
| 50 | Salesforce/blip2-opt-6.7b-coco open weights, 7.8B | 7.35% | huggingface.co | Official leaderboard | ||
| 51 | Salesforce/blip-image-captioning-large open weights, 0.4B | 6.57% | huggingface.co | Official leaderboard | ||
| 52 | Salesforce/blip-image-captioning-base open weights, 0.2B | 5.70% | huggingface.co | Official leaderboard |
| # | Model | Score | Setting | Date | Link | Source |
|---|---|---|---|---|---|---|
| 1 | | 22% | Oct 2026 | huggingface.co | Official leaderboard |
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: MTEB’s leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Vision
- MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy.
Qwen 3.6 Plus 86%
- MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
- CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
- MathVista In use Math reasoning in visual contexts (charts, geometry, puzzles) from 31 datasets. Answer accuracy.
Seed2.1 Pro 90.7%
- MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy.
Qwen 3.8 Max 97.7%
- MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy.
Qwen 3.8 Max 85.9%
Data audit
All 10 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (52 entries).
- Pass Models scored 1 model we track, plus 51 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for EmbeddingGemma 2.
- Note Frontier models None of its models are on the capabilities index.
- Pass Headroom The best is 22%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on MTEB: 1 row for models we track and 51 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.