Benchmark Almanac

MMEB

Qwen3-VL-Embedding-8B 80.1%
Not in model cards

Multimodal embedding models are scored on classification, visual question answering, multimodal retrieval and visual grounding, with 16 evaluation datasets in and out of distribution.

Introduced with the VLM2Vec model, ICLR 2025. 36 datasets in total, 20 for training and 16 for evaluation. Repo has 701 stars.

Released
October 2024
Kind
General visual questions
Level
Everyday
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Last model added
January 2026
Scores refreshed
November 12th, 2026
79
Entries on its own leaderboard, 2 models we track

Qwen3-VL-Embedding-8B leads with 80.1%

Best score per entry from the benchmark's own leaderboard, read on November 12th, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

79 entries. Scroll sideways for the rest.

  1. 80.1 Qwen3-VL-Embedding-8B
  2. 78.2 QQMM-embed-v3
  3. 78.1 WeMM-Embedding-8B
  4. 78.0 seed1.6-embedding-1215
  5. 77.9 IFM-TTE-7B
  6. 77.8 seed-1.6-embedding
  7. 76.1 WeMM-Embedding-2B
  8. 75.9 RzenEmbed-v2-7B
  9. 75.3 QQMM-embed-v2
  10. 75.0 Qwen3-VL-Embedding-2B
  11. 74.1 UniVec-CoT-7B
  12. 74.0 OEmbedding-v1-7B
  13. 73.9 ReCo-7B
  14. 73.8 UniVec-7B
  15. 73.7 ReMatch-7B
  16. 73.6 RzenEmbed-v1-7B
  17. 73.4 dp-embedding-v3-lite
  18. 72.7 Ops-MM-embedding-v1-7B
  19. 72.4 TCE-v1
  20. 72.2 QQMM-embed
  21. 72.0 B3_Qwen2_7B
  22. 71.8 UniME-V2-LLaVA-OneVision-7B
  23. 71.3 UME-R1-7B
  24. 71.2 e5-omni-7B
  25. 70.8 Embed-RL-4B
  26. 70.7 UniME(LLaVA-OneVision-7B-LoRA-Res336)
  27. 70.3 LLaVE-7B
  28. 70.3 UNITE-Instruct-7B
  29. 70.2 ReMatch-3B
  30. 69.8 interestFM-UIR-CAFe-7B
  31. 69.8 mmE5-mllama-11b-instruct
  32. 69.4 BGE-VL-v1.5 (FT; LlaVA-1.6-Mistral)
  33. 69.2 ReMatch-2B
  34. 69.0 Ops-MM-embedding-v1-2B
  35. 68.9 Embed-RL-2B
  36. 68.5 RzenEmbed-v1-2B
  37. 68.1 B3_Qwen2_2B
  38. 67.6 e5-omni-3B
  39. 66.6 UniME(LLaVA-1.6-7B-LoRA-LowRes)
  40. 66.6 UME-R1-2B
  41. 65.8 VLM2Vec (Qwen2-VL-7B-LoRA-HighRes)
  42. 65.4 VLM2Vec-V1-Qwen2VL-7B
  43. 65.2 LLaVE-2B
  44. 64.9 VLM2Vec-V2.0-Qwen2VL-2B
  45. 64.2 UniME(Phi-3.5-V-LoRA)
  46. 64.1 MMRet-MLLM (FT)
  47. 63.3 UNITE-Instruct-2B
  48. 62.9 VLM2Vec (LLaVA-1.6-LoRA-HighRes)
  49. 60.1 BGE-VL-v1.5 (zeroshot; LlaVA-1.6-Mistral)
  50. 60.1 VLM2Vec (Phi-3.5-V-LoRA)
  51. 59.7 VLM2Vec-V1-Qwen2VL-2B
  52. 59.6 interestFM-UIR-CAFe-0.5B
  53. 59.3 VLM2Vec (Qwen2-VL-2B-LoRA-HighRes)
  54. 59.1 LLaVE-0.5B
  55. 58.6 mmE5 (w/ 560K synthetic data)
  56. 55.9 gme-Qwen2-VL-7B-Instruct
  57. 55.9 VLM2Vec (Phi-3.5-V-FT)
  58. 55.8 gme-Qwen2-VL-2B-Instruct
  59. 55.0 VLM2Vec (LLaVA-1.6-LoRA-LowRes)
  60. 54.1 LamRA-Ret
  61. 52.5 LamRA-Ret-Qwen2.5VL-7b
  62. 50.0 MM-Embed
  63. 47.2 OpenCLIP-FT
  64. 45.4 CLIP-FT
  65. 44.7 UniIR (CLIP_SF)
  66. 44.0 MMRet-MLLM (LLaVA-1.6)
  67. 42.8 UniIR (BLIP_FF)
  68. 39.7 open_clip-ViT-L/14
  69. 37.8 clip-vit-large-patch14
  70. 34.9 colpali-v1.3
  71. 34.8 siglip-base-patch16-224
  72. 27.8 Magiclens
  73. 25.2 blip2-opt-2.7b
  74. 13.3 e5-v
  75. 0.0 Taichu-UniRetriever-2B
  76. 0.0 Taichu-UniRetriever-8B
  77. 0.0 TCR
  78. 0.0 Ops-Colqwen3-4B
  79. 0.0 GVE-7B
  1. 1 Qwen3-VL-Embedding-8B 80.1%
  2. 2 QQMM-embed-v3 78.2%
  3. 3 WeMM-Embedding-8B 78.1%
  4. 4 seed1.6-embedding-1215 78%
  5. 5 IFM-TTE-7B 77.9%
  6. 6 seed-1.6-embedding 77.8%
  7. 7 WeMM-Embedding-2B 76.1%
  8. 8 RzenEmbed-v2-7B 75.9%
  9. 9 QQMM-embed-v2 75.3%
  10. 10 Qwen3-VL-Embedding-2B 75%
  11. 11 UniVec-CoT-7B 74.1%
  12. 12 OEmbedding-v1-7B 74%
  13. 13 ReCo-7B 73.9%
  14. 14 UniVec-7B 73.8%
  15. 15 ReMatch-7B 73.7%
  16. 16 RzenEmbed-v1-7B 73.6%
  17. 17 dp-embedding-v3-lite 73.4%
  18. 18 Ops-MM-embedding-v1-7B 72.7%
  19. 19 TCE-v1 72.4%
  20. 20 QQMM-embed 72.2%
  21. 21 B3_Qwen2_7B 72%
  22. 22 UniME-V2-LLaVA-OneVision-7B 71.8%
  23. 23 UME-R1-7B 71.3%
  24. 24 e5-omni-7B 71.2%
  25. 25 Embed-RL-4B 70.8%
  26. 26 UniME(LLaVA-OneVision-7B-LoRA-Res336) 70.7%
  27. 27 LLaVE-7B 70.3%
  28. 28 UNITE-Instruct-7B 70.3%
  29. 29 ReMatch-3B 70.2%
  30. 30 interestFM-UIR-CAFe-7B 69.8%
  31. 31 mmE5-mllama-11b-instruct 69.8%
  32. 32 BGE-VL-v1.5 (FT; LlaVA-1.6-Mistral) 69.4%
  33. 33 ReMatch-2B 69.2%
  34. 34 Ops-MM-embedding-v1-2B 69%
  35. 35 Embed-RL-2B 68.9%
  36. 36 RzenEmbed-v1-2B 68.5%
  37. 37 B3_Qwen2_2B 68.1%
  38. 38 e5-omni-3B 67.6%
  39. 39 UniME(LLaVA-1.6-7B-LoRA-LowRes) 66.6%
  40. 40 UME-R1-2B 66.6%
  41. 41 VLM2Vec (Qwen2-VL-7B-LoRA-HighRes) 65.8%
  42. 42 VLM2Vec-V1-Qwen2VL-7B 65.4%
  43. 43 LLaVE-2B 65.2%
  44. 44 VLM2Vec-V2.0-Qwen2VL-2B 64.9%
  45. 45 UniME(Phi-3.5-V-LoRA) 64.2%
  46. 46 MMRet-MLLM (FT) 64.1%
  47. 47 UNITE-Instruct-2B 63.3%
  48. 48 VLM2Vec (LLaVA-1.6-LoRA-HighRes) 62.9%
  49. 49 BGE-VL-v1.5 (zeroshot; LlaVA-1.6-Mistral) 60.1%
  50. 50 VLM2Vec (Phi-3.5-V-LoRA) 60.1%
  51. 51 VLM2Vec-V1-Qwen2VL-2B 59.7%
  52. 52 interestFM-UIR-CAFe-0.5B 59.6%
  53. 53 VLM2Vec (Qwen2-VL-2B-LoRA-HighRes) 59.3%
  54. 54 LLaVE-0.5B 59.1%
  55. 55 mmE5 (w/ 560K synthetic data) 58.6%
  56. 56 gme-Qwen2-VL-7B-Instruct 55.9%
  57. 57 VLM2Vec (Phi-3.5-V-FT) 55.9%
  58. 58 gme-Qwen2-VL-2B-Instruct 55.8%
  59. 59 VLM2Vec (LLaVA-1.6-LoRA-LowRes) 55%
  60. 60 LamRA-Ret 54.1%
  61. 61 LamRA-Ret-Qwen2.5VL-7b 52.5%
  62. 62 MM-Embed 50%
  63. 63 OpenCLIP-FT 47.2%
  64. 64 CLIP-FT 45.4%
  65. 65 UniIR (CLIP_SF) 44.7%
  66. 66 MMRet-MLLM (LLaVA-1.6) 44%
  67. 67 UniIR (BLIP_FF) 42.8%
  68. 68 open_clip-ViT-L/14 39.7%
  69. 69 clip-vit-large-patch14 37.8%
  70. 70 colpali-v1.3 34.9%
  71. 71 siglip-base-patch16-224 34.8%
  72. 72 Magiclens 27.8%
  73. 73 blip2-opt-2.7b 25.2%
  74. 74 e5-v 13.3%
  75. 75 Taichu-UniRetriever-2B 0%
  76. 76 Taichu-UniRetriever-8B 0%
  77. 77 TCR 0%
  78. 78 Ops-Colqwen3-4B 0%
  79. 79 GVE-7B 0%

Source: Official MMEB leaderboard

#EntryScoreSettingDateSource
1 Qwen3-VL-Embedding-8B 80.1% Jan 2026 Official leaderboard
2 QQMM-embed-v3 78.2% Dec 2025 Official leaderboard
3 WeMM-Embedding-8B 78.1% Dec 2025 Official leaderboard
4 seed1.6-embedding-1215 78% Dec 2025 Official leaderboard
5 IFM-TTE-7B 77.9% Oct 2025 Official leaderboard
6 seed-1.6-embedding 77.8% Jun 2025 Official leaderboard
7 WeMM-Embedding-2B 76.1% Dec 2025 Official leaderboard
8 RzenEmbed-v2-7B 75.9% Sep 2025 Official leaderboard
9 QQMM-embed-v2 75.3% Sep 2025 Official leaderboard
10 Qwen3-VL-Embedding-2B 75% Jan 2026 Official leaderboard
11 UniVec-CoT-7B 74.1% unknown Official leaderboard
12 OEmbedding-v1-7B 74% Oct 2025 Official leaderboard
13 ReCo-7B 73.9% Aug 2025 Official leaderboard
14 UniVec-7B 73.8% unknown Official leaderboard
15 ReMatch-7B 73.7% Nov 2025 Official leaderboard
16 RzenEmbed-v1-7B 73.6% Jul 2025 Official leaderboard
17 dp-embedding-v3-lite 73.4% Dec 2025 Official leaderboard
18 Ops-MM-embedding-v1-7B 72.7% Jul 2025 Official leaderboard
19 TCE-v1 72.4% Oct 2025 Official leaderboard
20 QQMM-embed 72.2% unknown Official leaderboard
21 B3_Qwen2_7B 72% unknown Official leaderboard
22 UniME-V2-LLaVA-OneVision-7B 71.8% Oct 2025 Official leaderboard
23 UME-R1-7B 71.3% Oct 2025 Official leaderboard
24 e5-omni-7B 71.2% Jan 2026 Official leaderboard
25 Embed-RL-4B 70.8% Jan 2026 Official leaderboard
26 UniME(LLaVA-OneVision-7B-LoRA-Res336) 70.7% unknown Official leaderboard
27 LLaVE-7B 70.3% unknown Official leaderboard
28 UNITE-Instruct-7B 70.3% unknown Official leaderboard
29 ReMatch-3B 70.2% Nov 2025 Official leaderboard
30 interestFM-UIR-CAFe-7B 69.8% unknown Official leaderboard
31 mmE5-mllama-11b-instruct 69.8% unknown Official leaderboard
32 BGE-VL-v1.5 (FT; LlaVA-1.6-Mistral) 69.4% unknown Official leaderboard
33 ReMatch-2B 69.2% Nov 2025 Official leaderboard
34 Ops-MM-embedding-v1-2B 69% Jul 2025 Official leaderboard
35 Embed-RL-2B 68.9% Jan 2026 Official leaderboard
36 RzenEmbed-v1-2B 68.5% Jul 2025 Official leaderboard
37 B3_Qwen2_2B 68.1% unknown Official leaderboard
38 e5-omni-3B 67.6% Jan 2026 Official leaderboard
39 UniME(LLaVA-1.6-7B-LoRA-LowRes) 66.6% unknown Official leaderboard
40 UME-R1-2B 66.6% Oct 2025 Official leaderboard
41 VLM2Vec (Qwen2-VL-7B-LoRA-HighRes) 65.8% unknown Official leaderboard
42 VLM2Vec-V1-Qwen2VL-7B 65.4% Dec 2025 Official leaderboard
43 LLaVE-2B 65.2% unknown Official leaderboard
44 VLM2Vec-V2.0-Qwen2VL-2B 64.9% Dec 2025 Official leaderboard
45 UniME(Phi-3.5-V-LoRA) 64.2% unknown Official leaderboard
46 MMRet-MLLM (FT) 64.1% unknown Official leaderboard
47 UNITE-Instruct-2B 63.3% unknown Official leaderboard
48 VLM2Vec (LLaVA-1.6-LoRA-HighRes) 62.9% unknown Official leaderboard
49 BGE-VL-v1.5 (zeroshot; LlaVA-1.6-Mistral) 60.1% unknown Official leaderboard
50 VLM2Vec (Phi-3.5-V-LoRA) 60.1% unknown Official leaderboard
51 VLM2Vec-V1-Qwen2VL-2B 59.7% Dec 2025 Official leaderboard
52 interestFM-UIR-CAFe-0.5B 59.6% unknown Official leaderboard
53 VLM2Vec (Qwen2-VL-2B-LoRA-HighRes) 59.3% unknown Official leaderboard
54 LLaVE-0.5B 59.1% unknown Official leaderboard
55 mmE5 (w/ 560K synthetic data) 58.6% unknown Official leaderboard
56 gme-Qwen2-VL-7B-Instruct 55.9% Dec 2025 Official leaderboard
57 VLM2Vec (Phi-3.5-V-FT) 55.9% unknown Official leaderboard
58 gme-Qwen2-VL-2B-Instruct 55.8% unknown Official leaderboard
59 VLM2Vec (LLaVA-1.6-LoRA-LowRes) 55% unknown Official leaderboard
60 LamRA-Ret 54.1% Dec 2025 Official leaderboard
61 LamRA-Ret-Qwen2.5VL-7b 52.5% Jun 2025 Official leaderboard
62 MM-Embed 50% unknown Official leaderboard
63 OpenCLIP-FT 47.2% unknown Official leaderboard
64 CLIP-FT 45.4% unknown Official leaderboard
65 UniIR (CLIP_SF) 44.7% unknown Official leaderboard
66 MMRet-MLLM (LLaVA-1.6) 44% unknown Official leaderboard
67 UniIR (BLIP_FF) 42.8% unknown Official leaderboard
68 open_clip-ViT-L/14 39.7% unknown Official leaderboard
69 clip-vit-large-patch14 37.8% unknown Official leaderboard
70 colpali-v1.3 34.9% Dec 2025 Official leaderboard
71 siglip-base-patch16-224 34.8% unknown Official leaderboard
72 Magiclens 27.8% unknown Official leaderboard
73 blip2-opt-2.7b 25.2% unknown Official leaderboard
74 e5-v 13.3% unknown Official leaderboard
75 Taichu-UniRetriever-2B 0% Jan 2026 Official leaderboard
76 Taichu-UniRetriever-8B 0% Jan 2026 Official leaderboard
77 TCR 0% Jan 2026 Official leaderboard
78 Ops-Colqwen3-4B 0% Jan 2026 Official leaderboard
79 GVE-7B 0% unknown Official leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Qwen3-VL-Embedding-8B 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Qwen3-VL-Embedding-8B

Source: Official MMEB leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

Replaced by MMEB-V2

More in Vision

All 281 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. GPT-5.6 Sol 84.6%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%