JSICK
Models judge entailment and semantic relatedness in Japanese sentence pairs. An additional stress test examines sensitivity to word order and case marking.
The llm-jp-eval harness runs this evaluation with Japanese prompts.
- Made by
- Hitomi Yanaka and Koji Mineshima
- Released
- May 2021
- Kind
- Reasoning in many languages
- Level
- School
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Subject
- Languages and literature
- Language
- Japanese
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
- Last model added
- October 2026
- Scores refreshed
- November 21st, 2026
Octen/Octen-Embedding-8B tops the leaderboard with 89.6% Qwen3-Embedding-8B leads with 89.4% among models released by October 2026
Best score per entry from the benchmark's own leaderboard, read on November 21st, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Qwen3-Embedding-8B, at 89.4%. This view shows only models we track, each at its best.
The newest model with a score here came out in October 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6.2 Astra
- Claude Fable 5.2
-
Gemini 4 Carbon
- GPT-6.1 Terra
- and 16 more
- 89.6 Octen/Octen-Embedding-8B
- 89.4
Qwen3-Embedding-8B
- 88.8
Qwen3-Embedding-4B
- 88.3 Bytedance/Seed1.6-embedding-1215
- 87.8 microsoft/harrier-oss-v1-27b
- 86.6
Qwen3-Embedding-0.6B
- 85.8 microsoft/harrier-oss-v1-0.6b
- 85.6 GritLM/GritLM-7B
- 85.1 NGA-KR/ja-embed-sts
- 85.1 codefuse-ai/F2LLM-v2-8B
- 85.1 sionic-ai/comsat-embed-ja-8b-preview
- 85.0 google/gemini-embedding-001
- 85.0 BidirLM/BidirLM-Omni-2.5B-Embedding
- 85.0 codefuse-ai/F2LLM-v2-4B
- 84.9 tencent/KaLM-Embedding-Gemma3-12B-2511
- 84.7 codefuse-ai/F2LLM-v2-1.7B
- 84.5 codefuse-ai/F2LLM-v2-14B
- 84.4 google/embeddinggemma-300m
- 84.3 Linq-AI-Research/Linq-Embed-Mistral
- 84.2
EmbeddingGemma 2
- 84.2 microsoft/harrier-oss-v1-270m
- 83.8 nvidia/llama-embed-nemotron-8b
- 83.8 Salesforce/SFR-Embedding-2_R
- 83.7 Salesforce/SFR-Embedding-Mistral
- 83.7 codefuse-ai/F2LLM-v2-0.6B
- 83.5 intfloat/e5-mistral-7b-instruct
- 83.5 Lajavaness/bilingual-embedding-large
- 83.3 google/text-multilingual-embedding-002
- 83.3 hotchpotch/bekko-embedding-v1-a25m
- 83.1 Gameselo/STS-multilingual-mpnet-base-v2
- 83.0 hotchpotch/bekko-embedding-v1-a8m
- 82.9 cl-nagoya/ruri-base
- 82.8 cl-nagoya/ruri-base-v2
- 82.7 cl-nagoya/ruri-large
- 82.6 cl-nagoya/ruri-large-v2
- 82.6 Cohere/Cohere-embed-multilingual-light-v3.0
- 82.6 ICT-TIME-and-Querit/BOOM_4B_v1
- 82.6 sbintuitions/sarashina-embedding-v2-1b
- 82.5 intfloat/multilingual-e5-large-instruct
- 82.5 BidirLM/BidirLM-1.7B-Embedding
- 82.4 codefuse-ai/F2LLM-v2-330M
- 82.1 BidirLM/BidirLM-1B-Embedding
- 82.1 ai-sage/Giga-Embeddings-instruct-3B-0826
- 82.1 manu/bge-m3-custom-fr
- 82.0 Lajavaness/bilingual-embedding-base
- 81.9 nomic-ai/nomic-embed-text-v2-moe
- 81.9 ICT-TIME-and-Querit/ICT-TIME-and-Querit-embedding-v1
- 81.7 ai-sage/Giga-Embeddings-instruct-10B-A1.8B-0826
- 81.6 HIT-TMG/KaLM-embedding-multilingual-mini-v1
- 81.6 codefuse-ai/F2LLM-v2-160M
- 81.6 Snowflake/snowflake-arctic-embed-l-v2.0
- 81.6 GritLM/GritLM-8x7B
- 81.5 jinaai/jina-embeddings-v5-omni-nano
- 81.5 jinaai/jina-embeddings-v5-text-nano
- 81.4 intfloat/multilingual-e5-small
- 81.4 cl-nagoya/ruri-v3-30m
- 81.3 Lajavaness/bilingual-embedding-small
- 81.3 jinaai/jina-embeddings-v5-omni-small
- 81.3 jinaai/jina-embeddings-v5-text-small
- 81.3 mongodb/voyage-large-2-instruct
- 81.2 openai/text-embedding-3-large
- 81.2 intfloat/multilingual-e5-base
- 81.0 HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1
- 80.9 sionic-ai/comsat-embed-ja-0.3b-preview
- 80.8 BidirLM/BidirLM-0.6B-Embedding
- 80.8 openai/text-embedding-3-small
- 80.7 Snowflake/snowflake-arctic-embed-m-v2.0
- 80.6 BidirLM/BidirLM-270M-Embedding
- 80.6 hotchpotch/bekko-static-embedding-v1
- 80.3 jinaai/jina-embeddings-v4
- 80.1 ibm-granite/granite-embedding-311m-multilingual-r2
- 80.1 Alibaba-NLP/gte-multilingual-base
- 79.8 intfloat/multilingual-e5-large
- 79.7 codefuse-ai/F2LLM-v2-80M
- 79.7 OrdalieTech/Solon-embeddings-large-0.1
- 79.7 nvidia/NV-Embed-v2
- 79.5 sbintuitions/sarashina-embedding-v1-1b
- 79.3 NovaSearch/jasper_en_vision_language_v1
- 79.3 cnmoro/static-nomic-384-pten-v2-st
- 79.3 ibm-granite/granite-embedding-97m-multilingual-r2
- 79.2 BAAI/bge-m3
- 78.8 shibing624/text2vec-base-multilingual
- 78.7 nvidia/NV-Embed-v1
- 78.7 sentence-transformers/static-similarity-mrl-multilingual-v1
- 78.3 mongodb/voyage-3.5
- 78.2 jinaai/jina-embeddings-v3
- 78.1 mongodb/voyage-3-lite
- 78.0 cl-nagoya/ruri-v3-310m
- 77.9 NovaSearch/stella_en_1.5B_v5
- 77.4 McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp-supervised
- 77.3 ibm-granite/granite-embedding-107m-multilingual
- 77.2 Cohere/Cohere-embed-multilingual-v3.0
- 77.2 ibm-granite/granite-embedding-278m-multilingual
- 77.0 sentence-transformers/LaBSE
- 76.5 Haon-Chen/speed-embedding-7b-instruct
- 76.2 minishlab/potion-multilingual-128M
- 76.2 ibm-granite/granite-embedding-125m-english
- 75.9 mongodb/voyage-multilingual-2
- 75.8 cl-nagoya/ruri-v3-70m
- 75.4 Alibaba-NLP/gte-Qwen2-1.5B-instruct
- 75.1 Omartificial-Intelligence-Space/Arabic-MiniLM-L12-v2-all-nli-triplet
- 74.4 ibm-granite/granite-embedding-30m-english
- 73.9 cl-nagoya/ruri-v3-130m
- 73.3 Omartificial-Intelligence-Space/Arabic-labse-Matryoshka
- 73.2 Alibaba-NLP/gte-Qwen2-7B-instruct
- 72.7 Omartificial-Intelligence-Space/Arabic-all-nli-triplet-Matryoshka
- 71.9 intfloat/e5-large-v2
- 71.8 sentence-transformers/all-MiniLM-L12-v2
- 71.8 mongodb/voyage-3
- 71.3 brahmairesearch/slx-v0.1
- 71.3 sentence-transformers/all-MiniLM-L6-v2
- 71.3 sentence-transformers/all-mpnet-base-v2
- 71.3 intfloat/e5-base-v2
- 70.8 zeroentropy/zembed-1
- 70.8 Alibaba-NLP/gte-Qwen1.5-7B-instruct
- 70.6 nomic-ai/nomic-embed-text-v1
- 69.2 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
- 69.1 nomic-ai/nomic-embed-text-v1.5
- 68.7 nomic-ai/nomic-embed-text-v1-unsupervised
- 68.5 ai-forever/ru-en-RoSBERTa
- 68.5 intfloat/e5-small-v2
- 68.4 Cohere/Cohere-embed-english-v3.0
- 67.7 nomic-ai/nomic-embed-text-v1-ablated
- 67.6 Mihaiii/Ivysaur
- 67.5 avsolatorio/GIST-all-MiniLM-L6-v2
- 67.4 omarelshehy/arabic-english-sts-matryoshka
- 67.1 Cohere/Cohere-embed-english-light-v3.0
- 66.5 McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp-unsup-simcse
- 66.1 Omartificial-Intelligence-Space/Arabic-mpnet-base-all-nli-triplet
- 65.8 thenlper/gte-small
- 65.6 manu/sentence_croissant_alpha_v0.2
- 65.5 avsolatorio/GIST-small-Embedding-v0
- 65.5 Jaume/gemma-2b-embeddings
- 65.2 DeepPavlov/rubert-base-cased-sentence
- 65.1 manu/sentence_croissant_alpha_v0.3
- 65.1 avsolatorio/NoInstruct-small-Embedding-v0
- 65.0 BAAI/bge-small-en-v1.5
- 64.9 Mihaiii/Bulbasaur
- 64.7 sentence-transformers/paraphrase-multilingual-mpnet-base-v2
- 64.6 thenlper/gte-large
- 64.4 thenlper/gte-base
- 64.2 abhinand/MedEmbed-small-v0.1
- 64.2 BAAI/bge-large-en-v1.5
- 64.0 manu/sentence_croissant_alpha_v0.4
- 63.9 Mihaiii/Squirtle
- 63.9 Mihaiii/Wartortle
- 63.8 BAAI/bge-base-en-v1.5
- 63.6 avsolatorio/GIST-large-Embedding-v0
- 63.5 Mihaiii/gte-micro-v4
- 62.9 Mihaiii/Venusaur
- 62.8 avsolatorio/GIST-Embedding-v0
- 62.8 WhereIsAI/UAE-Large-V1
- 62.8 minishlab/potion-base-4M
- 62.7 minishlab/potion-base-8M
- 62.7 mixedbread-ai/mxbai-embed-large-v1
- 62.5 jinaai/jina-embeddings-v2-small-en
- 62.4 malenia1/ternary-weight-embedding
- 62.3 minishlab/potion-base-2M
- 62.2 minishlab/M2V_base_glove_subword
- 62.1 minishlab/M2V_base_output
- 61.5 Snowflake/snowflake-arctic-embed-m
- 61.0 infgrad/stella-base-en-v2
- 60.4 Snowflake/snowflake-arctic-embed-m-v1.5
- 60.4 dwzhu/e5-base-4k
- 60.2 Mihaiii/gte-micro
- 60.1 Snowflake/snowflake-arctic-embed-m-long
- 59.8 Snowflake/snowflake-arctic-embed-l
- 59.5 izhx/udever-bloom-7b1
- 58.3 intfloat/e5-base
- 58.1 NovaSearch/stella_en_400M_v5
- 57.9 bigscience/sgpt-bloom-7b1-msmarco
- 57.8 izhx/udever-bloom-1b1
- 57.6 deepfile/embedder-100p
- 57.5 Snowflake/snowflake-arctic-embed-xs
- 56.7 intfloat/e5-large
- 56.4 izhx/udever-bloom-3b
- 55.8 deepvk/USER-base
- 55.0 sdadas/mmlw-e5-large
- 54.8 sdadas/mmlw-e5-base
- 52.8 intfloat/e5-small
- 52.5 Snowflake/snowflake-arctic-embed-s
- 51.7 DeepPavlov/rubert-base-cased
- 50.7 deepvk/deberta-v1-base
- 49.9 consciousAI/cai-lunaris-text-embeddings
- 46.1 sdadas/mmlw-e5-small
- 43.8 sergeyzh/rubert-tiny-turbo
- 43.0 Hum-Works/lodestone-base-4096-v1
- 41.0 cointegrated/rubert-tiny
- 40.8 sergeyzh/LaBSE-ru-turbo
- 40.6 cointegrated/rubert-tiny2
- 39.5 izhx/udever-bloom-560m
- 37.6 cointegrated/LaBSE-en-ru
- 36.4 Omartificial-Intelligence-Space/Marbert-all-nli-triplet-Matryoshka
- 35.3 ai-forever/sbert_large_nlu_ru
- 35.2 aari1995/German_Semantic_STS_V2
- 35.0 ai-forever/sbert_large_mt_nlu_ru
- 33.0 silma-ai/silma-embeddding-matryoshka-v0.1
- 32.3 Omartificial-Intelligence-Space/Arabert-all-nli-triplet-Matryoshka
- 31.9 jinaai/jina-embeddings-v2-base-en
- 29.5 DeepPavlov/distilrubert-small-cased-conversational
- 26.4 deepvk/USER-bge-m3
- 20.6 minishlab/M2V_base_glove
- 2.01 jinaai/jina-embedding-b-en-v1
- 2.01 sdadas/mmlw-roberta-large
- 2.01 jinaai/jina-embedding-s-en-v1
- 1.97 sdadas/mmlw-roberta-base
- -3.57 mteb/baseline-random-encoder
- 1 Octen/Octen-Embedding-8B 89.6%
- 2
Qwen3-Embedding-8B 89.4%
- 3
Qwen3-Embedding-4B 88.8%
- 4 Bytedance/Seed1.6-embedding-1215 88.3%
- 5 microsoft/harrier-oss-v1-27b 87.8%
- 6
Qwen3-Embedding-0.6B 86.6%
- 7 microsoft/harrier-oss-v1-0.6b 85.8%
- 8 GritLM/GritLM-7B 85.6%
- 9 NGA-KR/ja-embed-sts 85.1%
- 10 codefuse-ai/F2LLM-v2-8B 85.1%
- 11 sionic-ai/comsat-embed-ja-8b-preview 85.1%
- 12 google/gemini-embedding-001 85%
- 13 BidirLM/BidirLM-Omni-2.5B-Embedding 85%
- 14 codefuse-ai/F2LLM-v2-4B 85%
- 15 tencent/KaLM-Embedding-Gemma3-12B-2511 84.9%
- 16 codefuse-ai/F2LLM-v2-1.7B 84.7%
- 17 codefuse-ai/F2LLM-v2-14B 84.5%
- 18 google/embeddinggemma-300m 84.4%
- 19 Linq-AI-Research/Linq-Embed-Mistral 84.3%
- 20
EmbeddingGemma 2 84.2%
- 21 microsoft/harrier-oss-v1-270m 84.2%
- 22 nvidia/llama-embed-nemotron-8b 83.8%
- 23 Salesforce/SFR-Embedding-2_R 83.8%
- 24 Salesforce/SFR-Embedding-Mistral 83.7%
- 25 codefuse-ai/F2LLM-v2-0.6B 83.7%
- 26 intfloat/e5-mistral-7b-instruct 83.5%
- 27 Lajavaness/bilingual-embedding-large 83.5%
- 28 google/text-multilingual-embedding-002 83.3%
- 29 hotchpotch/bekko-embedding-v1-a25m 83.3%
- 30 Gameselo/STS-multilingual-mpnet-base-v2 83.1%
- 31 hotchpotch/bekko-embedding-v1-a8m 83%
- 32 cl-nagoya/ruri-base 82.9%
- 33 cl-nagoya/ruri-base-v2 82.8%
- 34 cl-nagoya/ruri-large 82.7%
- 35 cl-nagoya/ruri-large-v2 82.6%
- 36 Cohere/Cohere-embed-multilingual-light-v3.0 82.6%
- 37 ICT-TIME-and-Querit/BOOM_4B_v1 82.6%
- 38 sbintuitions/sarashina-embedding-v2-1b 82.6%
- 39 intfloat/multilingual-e5-large-instruct 82.5%
- 40 BidirLM/BidirLM-1.7B-Embedding 82.5%
- 41 codefuse-ai/F2LLM-v2-330M 82.4%
- 42 BidirLM/BidirLM-1B-Embedding 82.1%
- 43 ai-sage/Giga-Embeddings-instruct-3B-0826 82.1%
- 44 manu/bge-m3-custom-fr 82.1%
- 45 Lajavaness/bilingual-embedding-base 82%
- 46 nomic-ai/nomic-embed-text-v2-moe 81.9%
- 47 ICT-TIME-and-Querit/ICT-TIME-and-Querit-embedding-v1 81.9%
- 48 ai-sage/Giga-Embeddings-instruct-10B-A1.8B-0826 81.7%
- 49 HIT-TMG/KaLM-embedding-multilingual-mini-v1 81.6%
- 50 codefuse-ai/F2LLM-v2-160M 81.6%
- 51 Snowflake/snowflake-arctic-embed-l-v2.0 81.6%
- 52 GritLM/GritLM-8x7B 81.6%
- 53 jinaai/jina-embeddings-v5-omni-nano 81.5%
- 54 jinaai/jina-embeddings-v5-text-nano 81.5%
- 55 intfloat/multilingual-e5-small 81.4%
- 56 cl-nagoya/ruri-v3-30m 81.4%
- 57 Lajavaness/bilingual-embedding-small 81.3%
- 58 jinaai/jina-embeddings-v5-omni-small 81.3%
- 59 jinaai/jina-embeddings-v5-text-small 81.3%
- 60 mongodb/voyage-large-2-instruct 81.3%
- 61 openai/text-embedding-3-large 81.2%
- 62 intfloat/multilingual-e5-base 81.2%
- 63 HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1 81%
- 64 sionic-ai/comsat-embed-ja-0.3b-preview 80.9%
- 65 BidirLM/BidirLM-0.6B-Embedding 80.8%
- 66 openai/text-embedding-3-small 80.8%
- 67 Snowflake/snowflake-arctic-embed-m-v2.0 80.7%
- 68 BidirLM/BidirLM-270M-Embedding 80.6%
- 69 hotchpotch/bekko-static-embedding-v1 80.6%
- 70 jinaai/jina-embeddings-v4 80.3%
- 71 ibm-granite/granite-embedding-311m-multilingual-r2 80.1%
- 72 Alibaba-NLP/gte-multilingual-base 80.1%
- 73 intfloat/multilingual-e5-large 79.8%
- 74 codefuse-ai/F2LLM-v2-80M 79.7%
- 75 OrdalieTech/Solon-embeddings-large-0.1 79.7%
- 76 nvidia/NV-Embed-v2 79.7%
- 77 sbintuitions/sarashina-embedding-v1-1b 79.5%
- 78 NovaSearch/jasper_en_vision_language_v1 79.3%
- 79 cnmoro/static-nomic-384-pten-v2-st 79.3%
- 80 ibm-granite/granite-embedding-97m-multilingual-r2 79.3%
- 81 BAAI/bge-m3 79.2%
- 82 shibing624/text2vec-base-multilingual 78.8%
- 83 nvidia/NV-Embed-v1 78.7%
- 84 sentence-transformers/static-similarity-mrl-multilingual-v1 78.7%
- 85 mongodb/voyage-3.5 78.3%
- 86 jinaai/jina-embeddings-v3 78.2%
- 87 mongodb/voyage-3-lite 78.1%
- 88 cl-nagoya/ruri-v3-310m 78%
- 89 NovaSearch/stella_en_1.5B_v5 77.9%
- 90 McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp-supervised 77.4%
- 91 ibm-granite/granite-embedding-107m-multilingual 77.3%
- 92 Cohere/Cohere-embed-multilingual-v3.0 77.2%
- 93 ibm-granite/granite-embedding-278m-multilingual 77.2%
- 94 sentence-transformers/LaBSE 77%
- 95 Haon-Chen/speed-embedding-7b-instruct 76.5%
- 96 minishlab/potion-multilingual-128M 76.2%
- 97 ibm-granite/granite-embedding-125m-english 76.2%
- 98 mongodb/voyage-multilingual-2 75.9%
- 99 cl-nagoya/ruri-v3-70m 75.8%
- 100 Alibaba-NLP/gte-Qwen2-1.5B-instruct 75.4%
- 101 Omartificial-Intelligence-Space/Arabic-MiniLM-L12-v2-all-nli-triplet 75.1%
- 102 ibm-granite/granite-embedding-30m-english 74.4%
- 103 cl-nagoya/ruri-v3-130m 73.9%
- 104 Omartificial-Intelligence-Space/Arabic-labse-Matryoshka 73.3%
- 105 Alibaba-NLP/gte-Qwen2-7B-instruct 73.2%
- 106 Omartificial-Intelligence-Space/Arabic-all-nli-triplet-Matryoshka 72.7%
- 107 intfloat/e5-large-v2 71.9%
- 108 sentence-transformers/all-MiniLM-L12-v2 71.8%
- 109 mongodb/voyage-3 71.8%
- 110 brahmairesearch/slx-v0.1 71.3%
- 111 sentence-transformers/all-MiniLM-L6-v2 71.3%
- 112 sentence-transformers/all-mpnet-base-v2 71.3%
- 113 intfloat/e5-base-v2 71.3%
- 114 zeroentropy/zembed-1 70.8%
- 115 Alibaba-NLP/gte-Qwen1.5-7B-instruct 70.8%
- 116 nomic-ai/nomic-embed-text-v1 70.6%
- 117 sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 69.2%
- 118 nomic-ai/nomic-embed-text-v1.5 69.1%
- 119 nomic-ai/nomic-embed-text-v1-unsupervised 68.7%
- 120 ai-forever/ru-en-RoSBERTa 68.5%
- 121 intfloat/e5-small-v2 68.5%
- 122 Cohere/Cohere-embed-english-v3.0 68.4%
- 123 nomic-ai/nomic-embed-text-v1-ablated 67.7%
- 124 Mihaiii/Ivysaur 67.6%
- 125 avsolatorio/GIST-all-MiniLM-L6-v2 67.5%
- 126 omarelshehy/arabic-english-sts-matryoshka 67.4%
- 127 Cohere/Cohere-embed-english-light-v3.0 67.1%
- 128 McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp-unsup-simcse 66.5%
- 129 Omartificial-Intelligence-Space/Arabic-mpnet-base-all-nli-triplet 66.1%
- 130 thenlper/gte-small 65.8%
- 131 manu/sentence_croissant_alpha_v0.2 65.6%
- 132 avsolatorio/GIST-small-Embedding-v0 65.5%
- 133 Jaume/gemma-2b-embeddings 65.5%
- 134 DeepPavlov/rubert-base-cased-sentence 65.2%
- 135 manu/sentence_croissant_alpha_v0.3 65.1%
- 136 avsolatorio/NoInstruct-small-Embedding-v0 65.1%
- 137 BAAI/bge-small-en-v1.5 65%
- 138 Mihaiii/Bulbasaur 64.9%
- 139 sentence-transformers/paraphrase-multilingual-mpnet-base-v2 64.7%
- 140 thenlper/gte-large 64.6%
- 141 thenlper/gte-base 64.4%
- 142 abhinand/MedEmbed-small-v0.1 64.2%
- 143 BAAI/bge-large-en-v1.5 64.2%
- 144 manu/sentence_croissant_alpha_v0.4 64%
- 145 Mihaiii/Squirtle 63.9%
- 146 Mihaiii/Wartortle 63.9%
- 147 BAAI/bge-base-en-v1.5 63.8%
- 148 avsolatorio/GIST-large-Embedding-v0 63.6%
- 149 Mihaiii/gte-micro-v4 63.5%
- 150 Mihaiii/Venusaur 62.9%
- 151 avsolatorio/GIST-Embedding-v0 62.8%
- 152 WhereIsAI/UAE-Large-V1 62.8%
- 153 minishlab/potion-base-4M 62.8%
- 154 minishlab/potion-base-8M 62.7%
- 155 mixedbread-ai/mxbai-embed-large-v1 62.7%
- 156 jinaai/jina-embeddings-v2-small-en 62.5%
- 157 malenia1/ternary-weight-embedding 62.4%
- 158 minishlab/potion-base-2M 62.3%
- 159 minishlab/M2V_base_glove_subword 62.2%
- 160 minishlab/M2V_base_output 62.1%
- 161 Snowflake/snowflake-arctic-embed-m 61.5%
- 162 infgrad/stella-base-en-v2 61%
- 163 Snowflake/snowflake-arctic-embed-m-v1.5 60.4%
- 164 dwzhu/e5-base-4k 60.4%
- 165 Mihaiii/gte-micro 60.2%
- 166 Snowflake/snowflake-arctic-embed-m-long 60.1%
- 167 Snowflake/snowflake-arctic-embed-l 59.8%
- 168 izhx/udever-bloom-7b1 59.5%
- 169 intfloat/e5-base 58.3%
- 170 NovaSearch/stella_en_400M_v5 58.1%
- 171 bigscience/sgpt-bloom-7b1-msmarco 57.9%
- 172 izhx/udever-bloom-1b1 57.8%
- 173 deepfile/embedder-100p 57.6%
- 174 Snowflake/snowflake-arctic-embed-xs 57.5%
- 175 intfloat/e5-large 56.7%
- 176 izhx/udever-bloom-3b 56.4%
- 177 deepvk/USER-base 55.8%
- 178 sdadas/mmlw-e5-large 55%
- 179 sdadas/mmlw-e5-base 54.8%
- 180 intfloat/e5-small 52.8%
- 181 Snowflake/snowflake-arctic-embed-s 52.5%
- 182 DeepPavlov/rubert-base-cased 51.7%
- 183 deepvk/deberta-v1-base 50.7%
- 184 consciousAI/cai-lunaris-text-embeddings 49.9%
- 185 sdadas/mmlw-e5-small 46.1%
- 186 sergeyzh/rubert-tiny-turbo 43.8%
- 187 Hum-Works/lodestone-base-4096-v1 43%
- 188 cointegrated/rubert-tiny 41%
- 189 sergeyzh/LaBSE-ru-turbo 40.8%
- 190 cointegrated/rubert-tiny2 40.6%
- 191 izhx/udever-bloom-560m 39.5%
- 192 cointegrated/LaBSE-en-ru 37.6%
- 193 Omartificial-Intelligence-Space/Marbert-all-nli-triplet-Matryoshka 36.4%
- 194 ai-forever/sbert_large_nlu_ru 35.3%
- 195 aari1995/German_Semantic_STS_V2 35.2%
- 196 ai-forever/sbert_large_mt_nlu_ru 35%
- 197 silma-ai/silma-embeddding-matryoshka-v0.1 33%
- 198 Omartificial-Intelligence-Space/Arabert-all-nli-triplet-Matryoshka 32.3%
- 199 jinaai/jina-embeddings-v2-base-en 31.9%
- 200 DeepPavlov/distilrubert-small-cased-conversational 29.5%
- 201 deepvk/USER-bge-m3 26.4%
- 202 minishlab/M2V_base_glove 20.6%
- 203 jinaai/jina-embedding-b-en-v1 2.01%
- 204 sdadas/mmlw-roberta-large 2.01%
- 205 jinaai/jina-embedding-s-en-v1 2.01%
- 206 sdadas/mmlw-roberta-base 1.97%
- 207 mteb/baseline-random-encoder -3.57%
Source: Official JSICK leaderboard
- 1
Qwen3-Embedding-8B 89.4%
- 2
Qwen3-Embedding-4B 88.8%
- 3
Qwen3-Embedding-0.6B 86.6%
- 4
EmbeddingGemma 2 84.2%
Source: Official JSICK leaderboard
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Octen/Octen-Embedding-8B | 89.6% | Official leaderboard | ||
| 2 | | 89.4% | Jun 2025 | Official leaderboard | |
| 3 | | 88.8% | Jun 2025 | Official leaderboard | |
| 4 | Bytedance/Seed1.6-embedding-1215 | 88.3% | Official leaderboard | ||
| 5 | microsoft/harrier-oss-v1-27b | 87.8% | Official leaderboard | ||
| 6 | | 86.6% | Jun 2025 | Official leaderboard | |
| 7 | microsoft/harrier-oss-v1-0.6b | 85.8% | Official leaderboard | ||
| 8 | GritLM/GritLM-7B | 85.6% | Official leaderboard | ||
| 9 | NGA-KR/ja-embed-sts | 85.1% | Official leaderboard | ||
| 10 | codefuse-ai/F2LLM-v2-8B | 85.1% | Official leaderboard | ||
| 11 | sionic-ai/comsat-embed-ja-8b-preview | 85.1% | Official leaderboard | ||
| 12 | google/gemini-embedding-001 | 85% | Official leaderboard | ||
| 13 | BidirLM/BidirLM-Omni-2.5B-Embedding | 85% | Official leaderboard | ||
| 14 | codefuse-ai/F2LLM-v2-4B | 85% | Official leaderboard | ||
| 15 | tencent/KaLM-Embedding-Gemma3-12B-2511 | 84.9% | Official leaderboard | ||
| 16 | codefuse-ai/F2LLM-v2-1.7B | 84.7% | Official leaderboard | ||
| 17 | codefuse-ai/F2LLM-v2-14B | 84.5% | Official leaderboard | ||
| 18 | google/embeddinggemma-300m | 84.4% | Official leaderboard | ||
| 19 | Linq-AI-Research/Linq-Embed-Mistral | 84.3% | Official leaderboard | ||
| 20 | | 84.2% | Oct 2026 | Official leaderboard | |
| 21 | microsoft/harrier-oss-v1-270m | 84.2% | Official leaderboard | ||
| 22 | nvidia/llama-embed-nemotron-8b | 83.8% | Official leaderboard | ||
| 23 | Salesforce/SFR-Embedding-2_R | 83.8% | Official leaderboard | ||
| 24 | Salesforce/SFR-Embedding-Mistral | 83.7% | Official leaderboard | ||
| 25 | codefuse-ai/F2LLM-v2-0.6B | 83.7% | Official leaderboard | ||
| 26 | intfloat/e5-mistral-7b-instruct | 83.5% | Official leaderboard | ||
| 27 | Lajavaness/bilingual-embedding-large | 83.5% | Official leaderboard | ||
| 28 | google/text-multilingual-embedding-002 | 83.3% | Official leaderboard | ||
| 29 | hotchpotch/bekko-embedding-v1-a25m | 83.3% | Official leaderboard | ||
| 30 | Gameselo/STS-multilingual-mpnet-base-v2 | 83.1% | Official leaderboard | ||
| 31 | hotchpotch/bekko-embedding-v1-a8m | 83% | Official leaderboard | ||
| 32 | cl-nagoya/ruri-base | 82.9% | Official leaderboard | ||
| 33 | cl-nagoya/ruri-base-v2 | 82.8% | Official leaderboard | ||
| 34 | cl-nagoya/ruri-large | 82.7% | Official leaderboard | ||
| 35 | cl-nagoya/ruri-large-v2 | 82.6% | Official leaderboard | ||
| 36 | Cohere/Cohere-embed-multilingual-light-v3.0 | 82.6% | Official leaderboard | ||
| 37 | ICT-TIME-and-Querit/BOOM_4B_v1 | 82.6% | Official leaderboard | ||
| 38 | sbintuitions/sarashina-embedding-v2-1b | 82.6% | Official leaderboard | ||
| 39 | intfloat/multilingual-e5-large-instruct | 82.5% | Official leaderboard | ||
| 40 | BidirLM/BidirLM-1.7B-Embedding | 82.5% | Official leaderboard | ||
| 41 | codefuse-ai/F2LLM-v2-330M | 82.4% | Official leaderboard | ||
| 42 | BidirLM/BidirLM-1B-Embedding | 82.1% | Official leaderboard | ||
| 43 | ai-sage/Giga-Embeddings-instruct-3B-0826 | 82.1% | Official leaderboard | ||
| 44 | manu/bge-m3-custom-fr | 82.1% | Official leaderboard | ||
| 45 | Lajavaness/bilingual-embedding-base | 82% | Official leaderboard | ||
| 46 | nomic-ai/nomic-embed-text-v2-moe | 81.9% | Official leaderboard | ||
| 47 | ICT-TIME-and-Querit/ICT-TIME-and-Querit-embedding-v1 | 81.9% | Official leaderboard | ||
| 48 | ai-sage/Giga-Embeddings-instruct-10B-A1.8B-0826 | 81.7% | Official leaderboard | ||
| 49 | HIT-TMG/KaLM-embedding-multilingual-mini-v1 | 81.6% | Official leaderboard | ||
| 50 | codefuse-ai/F2LLM-v2-160M | 81.6% | Official leaderboard | ||
| 51 | Snowflake/snowflake-arctic-embed-l-v2.0 | 81.6% | Official leaderboard | ||
| 52 | GritLM/GritLM-8x7B | 81.6% | Official leaderboard | ||
| 53 | jinaai/jina-embeddings-v5-omni-nano | 81.5% | Official leaderboard | ||
| 54 | jinaai/jina-embeddings-v5-text-nano | 81.5% | Official leaderboard | ||
| 55 | intfloat/multilingual-e5-small | 81.4% | Official leaderboard | ||
| 56 | cl-nagoya/ruri-v3-30m | 81.4% | Official leaderboard | ||
| 57 | Lajavaness/bilingual-embedding-small | 81.3% | Official leaderboard | ||
| 58 | jinaai/jina-embeddings-v5-omni-small | 81.3% | Official leaderboard | ||
| 59 | jinaai/jina-embeddings-v5-text-small | 81.3% | Official leaderboard | ||
| 60 | mongodb/voyage-large-2-instruct | 81.3% | Official leaderboard | ||
| 61 | openai/text-embedding-3-large | 81.2% | Official leaderboard | ||
| 62 | intfloat/multilingual-e5-base | 81.2% | Official leaderboard | ||
| 63 | HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1 | 81% | Official leaderboard | ||
| 64 | sionic-ai/comsat-embed-ja-0.3b-preview | 80.9% | Official leaderboard | ||
| 65 | BidirLM/BidirLM-0.6B-Embedding | 80.8% | Official leaderboard | ||
| 66 | openai/text-embedding-3-small | 80.8% | Official leaderboard | ||
| 67 | Snowflake/snowflake-arctic-embed-m-v2.0 | 80.7% | Official leaderboard | ||
| 68 | BidirLM/BidirLM-270M-Embedding | 80.6% | Official leaderboard | ||
| 69 | hotchpotch/bekko-static-embedding-v1 | 80.6% | Official leaderboard | ||
| 70 | jinaai/jina-embeddings-v4 | 80.3% | Official leaderboard | ||
| 71 | ibm-granite/granite-embedding-311m-multilingual-r2 | 80.1% | Official leaderboard | ||
| 72 | Alibaba-NLP/gte-multilingual-base | 80.1% | Official leaderboard | ||
| 73 | intfloat/multilingual-e5-large | 79.8% | Official leaderboard | ||
| 74 | codefuse-ai/F2LLM-v2-80M | 79.7% | Official leaderboard | ||
| 75 | OrdalieTech/Solon-embeddings-large-0.1 | 79.7% | Official leaderboard | ||
| 76 | nvidia/NV-Embed-v2 | 79.7% | Official leaderboard | ||
| 77 | sbintuitions/sarashina-embedding-v1-1b | 79.5% | Official leaderboard | ||
| 78 | NovaSearch/jasper_en_vision_language_v1 | 79.3% | Official leaderboard | ||
| 79 | cnmoro/static-nomic-384-pten-v2-st | 79.3% | Official leaderboard | ||
| 80 | ibm-granite/granite-embedding-97m-multilingual-r2 | 79.3% | Official leaderboard | ||
| 81 | BAAI/bge-m3 | 79.2% | Official leaderboard | ||
| 82 | shibing624/text2vec-base-multilingual | 78.8% | Official leaderboard | ||
| 83 | nvidia/NV-Embed-v1 | 78.7% | Official leaderboard | ||
| 84 | sentence-transformers/static-similarity-mrl-multilingual-v1 | 78.7% | Official leaderboard | ||
| 85 | mongodb/voyage-3.5 | 78.3% | Official leaderboard | ||
| 86 | jinaai/jina-embeddings-v3 | 78.2% | Official leaderboard | ||
| 87 | mongodb/voyage-3-lite | 78.1% | Official leaderboard | ||
| 88 | cl-nagoya/ruri-v3-310m | 78% | Official leaderboard | ||
| 89 | NovaSearch/stella_en_1.5B_v5 | 77.9% | Official leaderboard | ||
| 90 | McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp-supervised | 77.4% | Official leaderboard | ||
| 91 | ibm-granite/granite-embedding-107m-multilingual | 77.3% | Official leaderboard | ||
| 92 | Cohere/Cohere-embed-multilingual-v3.0 | 77.2% | Official leaderboard | ||
| 93 | ibm-granite/granite-embedding-278m-multilingual | 77.2% | Official leaderboard | ||
| 94 | sentence-transformers/LaBSE | 77% | Official leaderboard | ||
| 95 | Haon-Chen/speed-embedding-7b-instruct | 76.5% | Official leaderboard | ||
| 96 | minishlab/potion-multilingual-128M | 76.2% | Official leaderboard | ||
| 97 | ibm-granite/granite-embedding-125m-english | 76.2% | Official leaderboard | ||
| 98 | mongodb/voyage-multilingual-2 | 75.9% | Official leaderboard | ||
| 99 | cl-nagoya/ruri-v3-70m | 75.8% | Official leaderboard | ||
| 100 | Alibaba-NLP/gte-Qwen2-1.5B-instruct | 75.4% | Official leaderboard | ||
| 101 | Omartificial-Intelligence-Space/Arabic-MiniLM-L12-v2-all-nli-triplet | 75.1% | Official leaderboard | ||
| 102 | ibm-granite/granite-embedding-30m-english | 74.4% | Official leaderboard | ||
| 103 | cl-nagoya/ruri-v3-130m | 73.9% | Official leaderboard | ||
| 104 | Omartificial-Intelligence-Space/Arabic-labse-Matryoshka | 73.3% | Official leaderboard | ||
| 105 | Alibaba-NLP/gte-Qwen2-7B-instruct | 73.2% | Official leaderboard | ||
| 106 | Omartificial-Intelligence-Space/Arabic-all-nli-triplet-Matryoshka | 72.7% | Official leaderboard | ||
| 107 | intfloat/e5-large-v2 | 71.9% | Official leaderboard | ||
| 108 | sentence-transformers/all-MiniLM-L12-v2 | 71.8% | Official leaderboard | ||
| 109 | mongodb/voyage-3 | 71.8% | Official leaderboard | ||
| 110 | brahmairesearch/slx-v0.1 | 71.3% | Official leaderboard | ||
| 111 | sentence-transformers/all-MiniLM-L6-v2 | 71.3% | Official leaderboard | ||
| 112 | sentence-transformers/all-mpnet-base-v2 | 71.3% | Official leaderboard | ||
| 113 | intfloat/e5-base-v2 | 71.3% | Official leaderboard | ||
| 114 | zeroentropy/zembed-1 | 70.8% | Official leaderboard | ||
| 115 | Alibaba-NLP/gte-Qwen1.5-7B-instruct | 70.8% | Official leaderboard | ||
| 116 | nomic-ai/nomic-embed-text-v1 | 70.6% | Official leaderboard | ||
| 117 | sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 | 69.2% | Official leaderboard | ||
| 118 | nomic-ai/nomic-embed-text-v1.5 | 69.1% | Official leaderboard | ||
| 119 | nomic-ai/nomic-embed-text-v1-unsupervised | 68.7% | Official leaderboard | ||
| 120 | ai-forever/ru-en-RoSBERTa | 68.5% | Official leaderboard | ||
| 121 | intfloat/e5-small-v2 | 68.5% | Official leaderboard | ||
| 122 | Cohere/Cohere-embed-english-v3.0 | 68.4% | Official leaderboard | ||
| 123 | nomic-ai/nomic-embed-text-v1-ablated | 67.7% | Official leaderboard | ||
| 124 | Mihaiii/Ivysaur | 67.6% | Official leaderboard | ||
| 125 | avsolatorio/GIST-all-MiniLM-L6-v2 | 67.5% | Official leaderboard | ||
| 126 | omarelshehy/arabic-english-sts-matryoshka | 67.4% | Official leaderboard | ||
| 127 | Cohere/Cohere-embed-english-light-v3.0 | 67.1% | Official leaderboard | ||
| 128 | McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp-unsup-simcse | 66.5% | Official leaderboard | ||
| 129 | Omartificial-Intelligence-Space/Arabic-mpnet-base-all-nli-triplet | 66.1% | Official leaderboard | ||
| 130 | thenlper/gte-small | 65.8% | Official leaderboard | ||
| 131 | manu/sentence_croissant_alpha_v0.2 | 65.6% | Official leaderboard | ||
| 132 | avsolatorio/GIST-small-Embedding-v0 | 65.5% | Official leaderboard | ||
| 133 | Jaume/gemma-2b-embeddings | 65.5% | Official leaderboard | ||
| 134 | DeepPavlov/rubert-base-cased-sentence | 65.2% | Official leaderboard | ||
| 135 | manu/sentence_croissant_alpha_v0.3 | 65.1% | Official leaderboard | ||
| 136 | avsolatorio/NoInstruct-small-Embedding-v0 | 65.1% | Official leaderboard | ||
| 137 | BAAI/bge-small-en-v1.5 | 65% | Official leaderboard | ||
| 138 | Mihaiii/Bulbasaur | 64.9% | Official leaderboard | ||
| 139 | sentence-transformers/paraphrase-multilingual-mpnet-base-v2 | 64.7% | Official leaderboard | ||
| 140 | thenlper/gte-large | 64.6% | Official leaderboard | ||
| 141 | thenlper/gte-base | 64.4% | Official leaderboard | ||
| 142 | abhinand/MedEmbed-small-v0.1 | 64.2% | Official leaderboard | ||
| 143 | BAAI/bge-large-en-v1.5 | 64.2% | Official leaderboard | ||
| 144 | manu/sentence_croissant_alpha_v0.4 | 64% | Official leaderboard | ||
| 145 | Mihaiii/Squirtle | 63.9% | Official leaderboard | ||
| 146 | Mihaiii/Wartortle | 63.9% | Official leaderboard | ||
| 147 | BAAI/bge-base-en-v1.5 | 63.8% | Official leaderboard | ||
| 148 | avsolatorio/GIST-large-Embedding-v0 | 63.6% | Official leaderboard | ||
| 149 | Mihaiii/gte-micro-v4 | 63.5% | Official leaderboard | ||
| 150 | Mihaiii/Venusaur | 62.9% | Official leaderboard | ||
| 151 | avsolatorio/GIST-Embedding-v0 | 62.8% | Official leaderboard | ||
| 152 | WhereIsAI/UAE-Large-V1 | 62.8% | Official leaderboard | ||
| 153 | minishlab/potion-base-4M | 62.8% | Official leaderboard | ||
| 154 | minishlab/potion-base-8M | 62.7% | Official leaderboard | ||
| 155 | mixedbread-ai/mxbai-embed-large-v1 | 62.7% | Official leaderboard | ||
| 156 | jinaai/jina-embeddings-v2-small-en | 62.5% | Official leaderboard | ||
| 157 | malenia1/ternary-weight-embedding | 62.4% | Official leaderboard | ||
| 158 | minishlab/potion-base-2M | 62.3% | Official leaderboard | ||
| 159 | minishlab/M2V_base_glove_subword | 62.2% | Official leaderboard | ||
| 160 | minishlab/M2V_base_output | 62.1% | Official leaderboard | ||
| 161 | Snowflake/snowflake-arctic-embed-m | 61.5% | Official leaderboard | ||
| 162 | infgrad/stella-base-en-v2 | 61% | Official leaderboard | ||
| 163 | Snowflake/snowflake-arctic-embed-m-v1.5 | 60.4% | Official leaderboard | ||
| 164 | dwzhu/e5-base-4k | 60.4% | Official leaderboard | ||
| 165 | Mihaiii/gte-micro | 60.2% | Official leaderboard | ||
| 166 | Snowflake/snowflake-arctic-embed-m-long | 60.1% | Official leaderboard | ||
| 167 | Snowflake/snowflake-arctic-embed-l | 59.8% | Official leaderboard | ||
| 168 | izhx/udever-bloom-7b1 | 59.5% | Official leaderboard | ||
| 169 | intfloat/e5-base | 58.3% | Official leaderboard | ||
| 170 | NovaSearch/stella_en_400M_v5 | 58.1% | Official leaderboard | ||
| 171 | bigscience/sgpt-bloom-7b1-msmarco | 57.9% | Official leaderboard | ||
| 172 | izhx/udever-bloom-1b1 | 57.8% | Official leaderboard | ||
| 173 | deepfile/embedder-100p | 57.6% | Official leaderboard | ||
| 174 | Snowflake/snowflake-arctic-embed-xs | 57.5% | Official leaderboard | ||
| 175 | intfloat/e5-large | 56.7% | Official leaderboard | ||
| 176 | izhx/udever-bloom-3b | 56.4% | Official leaderboard | ||
| 177 | deepvk/USER-base | 55.8% | Official leaderboard | ||
| 178 | sdadas/mmlw-e5-large | 55% | Official leaderboard | ||
| 179 | sdadas/mmlw-e5-base | 54.8% | Official leaderboard | ||
| 180 | intfloat/e5-small | 52.8% | Official leaderboard | ||
| 181 | Snowflake/snowflake-arctic-embed-s | 52.5% | Official leaderboard | ||
| 182 | DeepPavlov/rubert-base-cased | 51.7% | Official leaderboard | ||
| 183 | deepvk/deberta-v1-base | 50.7% | Official leaderboard | ||
| 184 | consciousAI/cai-lunaris-text-embeddings | 49.9% | Official leaderboard | ||
| 185 | sdadas/mmlw-e5-small | 46.1% | Official leaderboard | ||
| 186 | sergeyzh/rubert-tiny-turbo | 43.8% | Official leaderboard | ||
| 187 | Hum-Works/lodestone-base-4096-v1 | 43% | Official leaderboard | ||
| 188 | cointegrated/rubert-tiny | 41% | Official leaderboard | ||
| 189 | sergeyzh/LaBSE-ru-turbo | 40.8% | Official leaderboard | ||
| 190 | cointegrated/rubert-tiny2 | 40.6% | Official leaderboard | ||
| 191 | izhx/udever-bloom-560m | 39.5% | Official leaderboard | ||
| 192 | cointegrated/LaBSE-en-ru | 37.6% | Official leaderboard | ||
| 193 | Omartificial-Intelligence-Space/Marbert-all-nli-triplet-Matryoshka | 36.4% | Official leaderboard | ||
| 194 | ai-forever/sbert_large_nlu_ru | 35.3% | Official leaderboard | ||
| 195 | aari1995/German_Semantic_STS_V2 | 35.2% | Official leaderboard | ||
| 196 | ai-forever/sbert_large_mt_nlu_ru | 35% | Official leaderboard | ||
| 197 | silma-ai/silma-embeddding-matryoshka-v0.1 | 33% | Official leaderboard | ||
| 198 | Omartificial-Intelligence-Space/Arabert-all-nli-triplet-Matryoshka | 32.3% | Official leaderboard | ||
| 199 | jinaai/jina-embeddings-v2-base-en | 31.9% | Official leaderboard | ||
| 200 | DeepPavlov/distilrubert-small-cased-conversational | 29.5% | Official leaderboard | ||
| 201 | deepvk/USER-bge-m3 | 26.4% | Official leaderboard | ||
| 202 | minishlab/M2V_base_glove | 20.6% | Official leaderboard | ||
| 203 | jinaai/jina-embedding-b-en-v1 | 2.01% | Official leaderboard | ||
| 204 | sdadas/mmlw-roberta-large | 2.01% | Official leaderboard | ||
| 205 | jinaai/jina-embedding-s-en-v1 | 2.01% | Official leaderboard | ||
| 206 | sdadas/mmlw-roberta-base | 1.97% | Official leaderboard | ||
| 207 | mteb/baseline-random-encoder | -3.57% | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 89.4% | Jun 2025 | Official leaderboard | |
| 2 | | 88.8% | Jun 2025 | Official leaderboard | |
| 3 | | 86.6% | Jun 2025 | Official leaderboard | |
| 4 | | 84.2% | Oct 2026 | Official leaderboard |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Source: Official JSICK leaderboard
Qwen3-Embedding-8B on the official leaderboard passed 85% in June 2025
That was 49 months after its release. All the scores we have, one dot each, so a model tested at several settings appears more than once.
Source: Official JSICK leaderboard
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Multilingual
- MMMLU In use MMLU professionally translated into 14 languages. Multiple-choice accuracy. Claude Mythos Preview 92.7%
- WMT24++ In use English-to-55-language translation of WMT24 test data. Scored with automatic metrics such as MetricX or xCOMET.
K-EXAONE 90.5%
- MMLU-ProX In use MMLU-Pro translated into up to 29 languages with expert review. Multiple-choice accuracy per language.
Qwen3.7-Max 87%
- CLIcK In use Models answer 1,995 multiple-choice questions on Korean language and culture. Questions come from official Korean exams and textbooks in eleven categories.
Solar Open 2 90.7%
- Global MMLU Lite 3.0 In use Models answer 400 human-translated MMLU questions per language in 23 languages. Version 3.0 adds Czech, Hungarian, Oriya, Slovak and Tajik and updates Italian. Accuracy. Claude Opus 5 95.7%
- MMMB In use Models answer multiple-choice questions about images in six languages across 15 categories. The set has about 12,000 questions. LFM2.5-VL-3B 83%
Data audit
All 9 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
- Pass Independent runs Scores from the official leaderboard (207 entries).
- Pass Models scored 4 models we track, plus 203 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for EmbeddingGemma 2.
- Note Frontier models None of its models are on the capabilities index.
- Note Headroom Saturated. The best is 89.4%, past 85% since June 2025.
Leaderboard import 3 pass
- Pass Importer We read the board on mteb-leaderboard-backend.hf.space: 4 rows for models we track and 203 other entries at the last read.
- Pass Last read November 21st, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.