Benchmark Almanac

FreshStack

Oracle: Fusion (BM25, ...) (Nuggets) 0.76 tops its leaderboard Gemini Embedding 2 0.41 leads the models we track
Not in model cards

Retrieval systems find documentation that supports answers to technical questions. The benchmark measures how many required facts the retrieved passages cover.

The official leaderboard publishes 62 system entries with per-domain retrieval metrics. Databricks also reports FreshStack in its KARL paper.

Released
April 2025
Kind
Questions that need search
Level
Expert
Scored
Checked automatically
Answers
Short answer
Tasks
All public
Measures
Capability
Language
English
In and out
Text → Text
Interaction
Single turn
Items from
Real work and use
Refresh
One fixed set
Run by
Anyone
Last model added
October 2026
Scores refreshed
November 21st, 2026
62
Entries on its own leaderboard, 2 models we track

Oracle: Fusion (BM25, ...) (Nuggets) tops the leaderboard with 0.76 Gemini Embedding 2 leads with 0.41

Best score per entry from the benchmark's own leaderboard, read on November 21st, 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Gemini Embedding 2, at 0.41. This view shows only models we track, each at its best.

62 entries. Scroll sideways for the rest.

  1. 0.76 Oracle: Fusion (BM25, ...) (Nuggets)
  2. 0.66 Mixedbread Wholembed V4
  3. 0.61 Octen-Embedding-8B
  4. 0.60 pplx-v2-preview
  5. 0.59 Oracle: Voyage Large 2 (Nuggets)
  6. 0.57 Octen-Embedding-4B
  7. 0.57 Cohere Embed v5.0 (Pro)
  8. 0.57 Nemotron-3.5-Embed-8B
  9. 0.57 Nemotron-3-Embed-8B
  10. 0.56 Perplexity Embed v2 Context (9B)
  11. 0.56 Oracle: BM25 (Nuggets)
  12. 0.55 Oracle: BGE (Gemma-2) (Nuggets)
  13. 0.54 Fusion (BM25, BGE, E5, Voyage)
  14. 0.53 Cohere Embed v5.0 (Fast)
  15. 0.53 zembed-1
  16. 0.53 Qwen3-8B (Emb)
  17. 0.51 voyage-4-large
  18. 0.50 Perplexity Embed v2 Late (9B)
  19. 0.50 harrier-oss-v1-27b
  20. 0.50 Oracle: E5 (Mistral-7B) (Nuggets)
  21. 0.49 Qwen3-4B (Emb)
  22. 0.48 Stella-1.5B v5
  23. 0.47 Octen-Embedding-0.6B
  24. 0.47 Nemotron-3-Embed-1B
  25. 0.47 Nemotron-3.5-Embed-1B
  26. 0.44 Perplexity Embed v2 Context (0.6B)
  27. 0.44 Voyage Large 2
  28. 0.44 voyage-4-nano
  29. 0.43 Perplexity Embed v1 (4B)
  30. 0.43 Jina V5 (Text Small)
  31. 0.43 BGE (Gemma-2)
  32. 0.42 Jina V4 (Emb)
  33. 0.42 Stella-400M v5
  34. 0.42 Jina V5 (Text Nano)
  35. 0.41 Gemini Embedding 2
  36. 0.41 harrier-oss-v1-0.6b
  37. 0.40 Jasper-Token-Compression-600M
  38. 0.40 E5 (Mistral-7B)
  39. 0.40 Perplexity Embed v1 (0.6B)
  40. 0.39 Qwen3-0.6B (Emb)
  41. 0.39 Cohere Embed v4.0
  42. 0.39 Arctic Embed M (v2.0)
  43. 0.37 OpenAI text-embedding-3-large
  44. 0.36 Arctic Embed L (v2.0)
  45. 0.35 LateOn
  46. 0.35 Granite-Embedding-English-R2 (Small)
  47. 0.35 EmbeddingGemma 2
  48. 0.35 Nomic Embed (Code)
  49. 0.34 Tarka-Embedding-350M-V1
  50. 0.34 Jina V3 (Emb)
  51. 0.34 Granite-Embedding-English-R2
  52. 0.34 EmbeddingGemma-300M
  53. 0.33 OpenAI text-embedding-3-small
  54. 0.33 GTE (ModernBERT-base)
  55. 0.32 GTE (large) v1.5
  56. 0.32 BM25
  57. 0.31 DenseOn
  58. 0.29 harrier-oss-v1-270m
  59. 0.28 SlideOn
  60. 0.27 Tarka-Embedding-150M-V1
  61. 0.27 Perplexity Embed v2 Late (0.6B)
  62. 0.16 CodeRankEmbed
  1. 1 Oracle: Fusion (BM25, ...) (Nuggets) 0.76
  2. 2 Mixedbread Wholembed V4 0.66
  3. 3 Octen-Embedding-8B 0.61
  4. 4 pplx-v2-preview 0.60
  5. 5 Oracle: Voyage Large 2 (Nuggets) 0.59
  6. 6 Octen-Embedding-4B 0.57
  7. 7 Cohere Embed v5.0 (Pro) 0.57
  8. 8 Nemotron-3.5-Embed-8B 0.57
  9. 9 Nemotron-3-Embed-8B 0.57
  10. 10 Perplexity Embed v2 Context (9B) 0.56
  11. 11 Oracle: BM25 (Nuggets) 0.56
  12. 12 Oracle: BGE (Gemma-2) (Nuggets) 0.55
  13. 13 Fusion (BM25, BGE, E5, Voyage) 0.54
  14. 14 Cohere Embed v5.0 (Fast) 0.53
  15. 15 zembed-1 0.53
  16. 16 Qwen3-8B (Emb) 0.53
  17. 17 voyage-4-large 0.51
  18. 18 Perplexity Embed v2 Late (9B) 0.50
  19. 19 harrier-oss-v1-27b 0.50
  20. 20 Oracle: E5 (Mistral-7B) (Nuggets) 0.50
  21. 21 Qwen3-4B (Emb) 0.49
  22. 22 Stella-1.5B v5 0.48
  23. 23 Octen-Embedding-0.6B 0.47
  24. 24 Nemotron-3-Embed-1B 0.47
  25. 25 Nemotron-3.5-Embed-1B 0.47
  26. 26 Perplexity Embed v2 Context (0.6B) 0.44
  27. 27 Voyage Large 2 0.44
  28. 28 voyage-4-nano 0.44
  29. 29 Perplexity Embed v1 (4B) 0.43
  30. 30 Jina V5 (Text Small) 0.43
  31. 31 BGE (Gemma-2) 0.43
  32. 32 Jina V4 (Emb) 0.42
  33. 33 Stella-400M v5 0.42
  34. 34 Jina V5 (Text Nano) 0.42
  35. 35 Gemini Embedding 2 0.41
  36. 36 harrier-oss-v1-0.6b 0.41
  37. 37 Jasper-Token-Compression-600M 0.40
  38. 38 E5 (Mistral-7B) 0.40
  39. 39 Perplexity Embed v1 (0.6B) 0.40
  40. 40 Qwen3-0.6B (Emb) 0.39
  41. 41 Cohere Embed v4.0 0.39
  42. 42 Arctic Embed M (v2.0) 0.39
  43. 43 OpenAI text-embedding-3-large 0.37
  44. 44 Arctic Embed L (v2.0) 0.36
  45. 45 LateOn 0.35
  46. 46 Granite-Embedding-English-R2 (Small) 0.35
  47. 47 EmbeddingGemma 2 0.35
  48. 48 Nomic Embed (Code) 0.35
  49. 49 Tarka-Embedding-350M-V1 0.34
  50. 50 Jina V3 (Emb) 0.34
  51. 51 Granite-Embedding-English-R2 0.34
  52. 52 EmbeddingGemma-300M 0.34
  53. 53 OpenAI text-embedding-3-small 0.33
  54. 54 GTE (ModernBERT-base) 0.33
  55. 55 GTE (large) v1.5 0.32
  56. 56 BM25 0.32
  57. 57 DenseOn 0.31
  58. 58 harrier-oss-v1-270m 0.29
  59. 59 SlideOn 0.28
  60. 60 Tarka-Embedding-150M-V1 0.27
  61. 61 Perplexity Embed v2 Late (0.6B) 0.27
  62. 62 CodeRankEmbed 0.16

Source: Official FreshStack leaderboard

Source: Official FreshStack leaderboard

#EntryScoreSettingReleasedRun onLinkSource
1 Oracle: Fusion (BM25, ...) (Nuggets) 0.76 Nov 2024 Official leaderboard
2 Mixedbread Wholembed V4 0.66 Oct 2026 mixedbread.com Official leaderboard
3 Octen-Embedding-8B 0.61 Jan 2026 huggingface.co Official leaderboard
4 pplx-v2-preview 0.60 Sep 2026 huggingface.co Official leaderboard
5 Oracle: Voyage Large 2 (Nuggets) 0.59 Nov 2024 docs.voyageai.com Official leaderboard
6 Octen-Embedding-4B 0.57 Jan 2026 huggingface.co Official leaderboard
7 Cohere Embed v5.0 (Pro) 0.57 Sep 2026 docs.cohere.com Official leaderboard
8 Nemotron-3.5-Embed-8B 0.57 Oct 2026 huggingface.co Official leaderboard
9 Nemotron-3-Embed-8B 0.57 Jul 2026 huggingface.co Official leaderboard
10 Perplexity Embed v2 Context (9B) 0.56 Oct 2026 huggingface.co Official leaderboard
11 Oracle: BM25 (Nuggets) 0.56 Nov 2024 github.com Official leaderboard
12 Oracle: BGE (Gemma-2) (Nuggets) 0.55 Nov 2024 huggingface.co Official leaderboard
13 Fusion (BM25, BGE, E5, Voyage) 0.54 Nov 2024 Official leaderboard
14 Cohere Embed v5.0 (Fast) 0.53 Sep 2026 docs.cohere.com Official leaderboard
15 zembed-1 0.53 Mar 2026 huggingface.co Official leaderboard
16 Qwen3-8B (Emb) 0.53 Jun 2025 huggingface.co Official leaderboard
17 voyage-4-large 0.51 Jan 2026 blog.voyageai.com Official leaderboard
18 Perplexity Embed v2 Late (9B) 0.50 Oct 2026 huggingface.co Official leaderboard
19 harrier-oss-v1-27b 0.50 Mar 2026 huggingface.co Official leaderboard
20 Oracle: E5 (Mistral-7B) (Nuggets) 0.50 Nov 2024 huggingface.co Official leaderboard
21 Qwen3-4B (Emb) 0.49 Jun 2025 huggingface.co Official leaderboard
22 Stella-1.5B v5 0.48 Jan 2025 huggingface.co Official leaderboard
23 Octen-Embedding-0.6B 0.47 Jan 2026 huggingface.co Official leaderboard
24 Nemotron-3-Embed-1B 0.47 Jul 2026 huggingface.co Official leaderboard
25 Nemotron-3.5-Embed-1B 0.47 Oct 2026 huggingface.co Official leaderboard
26 Perplexity Embed v2 Context (0.6B) 0.44 Oct 2026 huggingface.co Official leaderboard
27 Voyage Large 2 0.44 May 2024 docs.voyageai.com Official leaderboard
28 voyage-4-nano 0.44 Jan 2026 huggingface.co Official leaderboard
29 Perplexity Embed v1 (4B) 0.43 Feb 2026 huggingface.co Official leaderboard
30 Jina V5 (Text Small) 0.43 Feb 2026 huggingface.co Official leaderboard
31 BGE (Gemma-2) 0.43 Jul 2024 huggingface.co Official leaderboard
32 Jina V4 (Emb) 0.42 Jun 2025 huggingface.co Official leaderboard
33 Stella-400M v5 0.42 Jan 2025 huggingface.co Official leaderboard
34 Jina V5 (Text Nano) 0.42 Feb 2026 huggingface.co Official leaderboard
35 Gemini Embedding 2 0.41 Mar 2026 Mar 2026 blog.google Official leaderboard
36 harrier-oss-v1-0.6b 0.41 Mar 2026 huggingface.co Official leaderboard
37 Jasper-Token-Compression-600M 0.40 Nov 2025 huggingface.co Official leaderboard
38 E5 (Mistral-7B) 0.40 Jan 2024 huggingface.co Official leaderboard
39 Perplexity Embed v1 (0.6B) 0.40 Feb 2026 huggingface.co Official leaderboard
40 Qwen3-0.6B (Emb) 0.39 Jun 2025 huggingface.co Official leaderboard
41 Cohere Embed v4.0 0.39 May 2025 cohere.com Official leaderboard
42 Arctic Embed M (v2.0) 0.39 Dec 2024 huggingface.co Official leaderboard
43 OpenAI text-embedding-3-large 0.37 Jan 2024 openai.com Official leaderboard
44 Arctic Embed L (v2.0) 0.36 Dec 2024 huggingface.co Official leaderboard
45 LateOn 0.35 Mar 2026 huggingface.co Official leaderboard
46 Granite-Embedding-English-R2 (Small) 0.35 Jul 2024 huggingface.co Official leaderboard
47 EmbeddingGemma 2 0.35 Oct 2026 Oct 2026 huggingface.co Official leaderboard
48 Nomic Embed (Code) 0.35 Mar 2025 huggingface.co Official leaderboard
49 Tarka-Embedding-350M-V1 0.34 Nov 2025 huggingface.co Official leaderboard
50 Jina V3 (Emb) 0.34 Sep 2024 huggingface.co Official leaderboard
51 Granite-Embedding-English-R2 0.34 Jul 2024 huggingface.co Official leaderboard
52 EmbeddingGemma-300M 0.34 Sep 2025 developers.googleblog.com Official leaderboard
53 OpenAI text-embedding-3-small 0.33 Jan 2024 openai.com Official leaderboard
54 GTE (ModernBERT-base) 0.33 Jan 2025 huggingface.co Official leaderboard
55 GTE (large) v1.5 0.32 Jan 2024 huggingface.co Official leaderboard
56 BM25 0.32 Nov 2024 github.com Official leaderboard
57 DenseOn 0.31 Mar 2026 huggingface.co Official leaderboard
58 harrier-oss-v1-270m 0.29 Mar 2026 huggingface.co Official leaderboard
59 SlideOn 0.28 Oct 2026 huggingface.co Official leaderboard
60 Tarka-Embedding-150M-V1 0.27 Nov 2025 huggingface.co Official leaderboard
61 Perplexity Embed v2 Late (0.6B) 0.27 Oct 2026 huggingface.co Official leaderboard
62 CodeRankEmbed 0.16 Nov 2024 huggingface.co Official leaderboard
#ModelScoreSettingReleasedRun onLinkSource
1 Gemini Embedding 2 0.41 Mar 2026 Mar 2026 blog.google Official leaderboard
2 EmbeddingGemma 2 0.35 Oct 2026 Oct 2026 huggingface.co Official leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.34 0.36 0.38 0.40 0.42 2026 0.34 0.36 0.38 0.40 0.42 2026

Source: Official FreshStack leaderboard

No lab model card reports it yet

We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.

More in Knowledge

All 332 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • MMLU In use 57-subject multiple-choice exam questions from elementary to professional level. Scored by accuracy. Seed1.8 92.3%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • BBEH micro average In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is the share of all examples answered correctly. Gemma 4 31B IT 74.4%
  • HLE-Diamond In use Models answer 1,000 cleaned expert questions from Humanity's Last Exam, 500 on reasoning and 500 on knowledge, closed book. Graded on exact answers. Claude Fable 5.2 77.9%

Data audit

All 9 checks pass. Every benchmark

Catalog record 2 pass
  • Pass Task access All tasks are public.
  • Pass Confidence Checked against a primary source.
Scores 3 pass 2 notes
  • Pass Independent runs Scores from the official leaderboard (62 entries).
  • Pass Models scored 2 models we track, plus 60 other entries on its leaderboard.
  • Pass Newest score The newest is from October 2026, for EmbeddingGemma 2.
  • Note Frontier models None of its models are on the capabilities index.
  • Note Headroom Scored as a plain number, with no fixed ceiling.
Leaderboard import 3 pass
  • Pass Importer We read the board on fresh-stack.github.io: 2 rows for models we track and 60 other entries at the last read.
  • Pass Last read November 21st, 2026.
  • Pass Read worked The last read worked.
Consistency 1 pass 1 note
  • Note Lab claims No model has both a lab number and an independent run.
  • Pass Known issues No published audit found problems.

How we check