BFCL V4
Version V4
The official leaderboard hasn’t scored a model released after GPT-5.2 (December 2025). Labs report their own numbers for 27 newer models, such as Qwen3.7-Max at 75%.
Models call functions and tools, including multi-turn agentic tasks with web search and memory. Checked against expected calls and outcomes.
V4 added agentic web search and memory categories and changed the overall formula, so its scores do not compare with V3.
- Made by
- UC Berkeley (Gorilla)
- Released
- July 2025
- Announced on X
- 12K views, 55 likes
- Kind
- Function calling
- Level
- Everyday
- Scored
- Checked automatically
- Answers
- Agent task
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Agentic
- Environment
- APIs and tools, Browser
- Refresh
- New versions replace it
- Run by
- Anyone
- In views
- Agents
- Strongest model tested
-
Qwen3.7-Max #49 on the capability index
- Last in a launch post
- October 2026
- Last model added
- October 2026
- Scores refreshed
- November 20th, 2026
Qwen3.7-Max leads with 75%
Best score per model as labs report them in their own cards, drawn striped. Labs pick their own settings, so these compare less cleanly than independent runs.
Claude Opus 4.5 leads with 77.5% across all independent runs
Best score per model across the official leaderboard, Epoch AI’s own runs and any board Epoch AI copied. Dotted bars are reruns by Epoch AI or another board, and each row names its source.
Claude Opus 4.5 leads with 77.5%, claims included
Best score per model from every source, including labs’ own numbers and claims Epoch AI collected from reports and papers. Labs pick their own settings, so a claim can beat an independent run of the same model. Striped bars are claims, and dotted bars are reruns by Epoch AI or another board.
- 75.0
Qwen3.7-Max
- 73.4
Seed2.0 Pro
- 72.9
Qwen3.5 397B-A17B
- 72.9
Qwen3.7-Plus
- 72.2
Qwen3.5-122B-A10B
- 71.8
Spark-X2.6-4B
- 71.4 LFM3-1.7B
- 70.6
Ling-2.6-1T
- 70.1
Trinity Large Thinking
- 68.5
Qwen3.5-27B
- 67.7
Qwen3-Max-2026-01-23
- 67.3
Qwen3.5-35B-A3B
- 66.5
ERNIE 5.0
- 66.1
Qwen3.5-9B
- 65.7
Keye-VL-2.0-30B-A3B
- 65.0 LFM2.5-24B-A2B
- 63.3
Qwen3.5-Omni-Plus
- 61.6
Nova 2.0 Pro Preview
- 61.4 Granite 4.2 30B
- 60.3
Nova 2 Lite
- 60.1
MiniCPM-V-4.7-35B-A3B
- 58.3
Nova 2 Omni Preview
- 57.2
Seed1.8
- 56.9 LFM2.5-2.6B
- 53.8
Nemotron 3 Nano 30B-A3B
- 52.9
Nemotron-Cascade 2 30B-A3B
- 50.3
Qwen3.5-4B
- 48.5 LFM2.5-8B-A1B
- 43.6
Qwen3.5-2B
- 32.5 LFM2.5-VL-3B
- 25.3
Qwen3.5-0.8B
- 1
Qwen3.7-Max 75%
- 2
Seed2.0 Pro 73.4%
- 3
Qwen3.5 397B-A17B 72.9%
- 4
Qwen3.7-Plus 72.9%
- 5
Qwen3.5-122B-A10B 72.2%
- 6
Spark-X2.6-4B 71.8%
- 7 LFM3-1.7B 71.4%
- 8
Ling-2.6-1T 70.6%
- 9
Trinity Large Thinking 70.1%
- 10
Qwen3.5-27B 68.5%
- 11
Qwen3-Max-2026-01-23 67.7%
- 12
Qwen3.5-35B-A3B 67.3%
- 13
ERNIE 5.0 66.5%
- 14
Qwen3.5-9B 66.1%
- 15
Keye-VL-2.0-30B-A3B 65.7%
- 16 LFM2.5-24B-A2B 65%
- 17
Qwen3.5-Omni-Plus 63.3%
- 18
Nova 2.0 Pro Preview 61.6%
- 19 Granite 4.2 30B 61.4%
- 20
Nova 2 Lite 60.3%
- 21
MiniCPM-V-4.7-35B-A3B 60.1%
- 22
Nova 2 Omni Preview 58.3%
- 23
Seed1.8 57.2%
- 24 LFM2.5-2.6B 56.9%
- 25
Nemotron 3 Nano 30B-A3B 53.8%
- 26
Nemotron-Cascade 2 30B-A3B 52.9%
- 27
Qwen3.5-4B 50.3%
- 28 LFM2.5-8B-A1B 48.5%
- 29
Qwen3.5-2B 43.6%
- 30 LFM2.5-VL-3B 32.5%
- 31
Qwen3.5-0.8B 25.3%
Source: Labs’ model cards
- 77.5 Claude Opus 4.5
- 73.2 Claude Sonnet 4.5
- 72.5
Gemini 3 Pro
- 72.4 GLM-4.6
- 69.6 Grok 4.1 Fast
- 68.7 Claude Haiku 4.5
- 63.0 o3
- 63.0 Grok 4
- 59.1 Kimi K2
- 57.1
Command A Reasoning
- 56.7
DeepSeek-V3.2-Exp
- 56.2
Gemini 2.5 Flash
- 55.9 GPT-5.2
- 55.5 GPT-5 mini
- 54.7
xLAM-2-32b-fc-r
- 54.0 GPT-4.1
- 53.2 o4-mini
- 53.1
Llama-xLAM-2-70b-fc-r
- 52.1
Qwen3-235B-A22B-Instruct
- 51.5 GPT-5 nano
- 50.5 GPT-4.1 mini
- 48.7
Qwen3-32B
- 46.7
Llama-xLAM-2-8b-fc-r
- 46.5
Cohere Command A
- 42.6
Qwen3-8B
- 41.4
Qwen3-30B-A3B-Instruct
- 41.2
xLAM-2-3b-fc-r
- 41.0
Qwen3-14B
- 38.4
Mistral Large 2
- 37.7
Mistral Medium 3
- 37.3
Llama 4 Maverick
- 37.1
Mistral Small 3.2
- 36.9
Gemini 2.5 Flash-Lite
- 35.7 qwen3-4b-instruct-2507
- 33.0 GPT-4.1 nano
- 32.1
Command R7B
- 31.9
Llama 3.3 70B Instruct
- 30.4
xLAM-2-1b-fc-r
- 30.4
Gemma 3 12B
- 29.5
Gemma 3 27B
- 28.8
Phi-4
- 28.4
Qwen3-1.7B
- 28.1
Llama 4 Scout
- 27.9 Palmyra X4
- 27.6
Mistral NeMo
- 27.1 Granite-3.1-8B-Instruct
- 27.1
Nova 2 Lite
- 27.0
Falcon3-10B-Instruct
- 26.9 Granite 3.2 8B Instruct
- 25.8
Llama 3.1-8B
- 25.6
MiniCPM3-4B
- 25.0
Amazon Nova Pro
- 24.0
Falcon3-7B-Instruct
- 23.9
Qwen3-0.6B
- 23.2 Granite-20B-FunctionCalling
- 22.3
Amazon Nova Micro
- 21.9
Llama 3.2 3B
- 19.6
Gemma 3 4B
- 19.0 granite-4.0-350m
- 16.3
Falcon3-3B-Instruct
- 11.1
Ministral 8B
- 11.1
Falcon3-1B-Instruct
- 10.8
Llama 3.2 1B
- 10.0
Llama-3.1-Nemotron-Ultra-253B-v1
- 7.17
Gemma 3 1B
- 1 Claude Opus 4.5 native function calling 77.5%
- 2 Claude Sonnet 4.5 native function calling 73.2%
- 3
Gemini 3 Pro prompted 72.5%
- 4 GLM-4.6 thinking; native function calling 72.4%
- 5 Grok 4.1 Fast reasoning; native function calling 69.6%
- 6 Claude Haiku 4.5 native function calling 68.7%
- 7 o3 prompted 63%
- 8 Grok 4 prompted 63%
- 9 Kimi K2 Jul 2025, native function calling 59.1%
- 10
Command A Reasoning native function calling 57.1%
- 11
DeepSeek-V3.2-Exp thinking; prompted 56.7%
- 12
Gemini 2.5 Flash Sep 2025, native function calling 56.2%
- 13 GPT-5.2 native function calling 55.9%
- 14 GPT-5 mini native function calling 55.5%
- 15
xLAM-2-32b-fc-r native function calling 54.7%
- 16 GPT-4.1 native function calling 54%
- 17 o4-mini native function calling 53.2%
- 18
Llama-xLAM-2-70b-fc-r native function calling 53.1%
- 19
Qwen3-235B-A22B-Instruct Jul 2025, prompted 52.1%
- 20 GPT-5 nano native function calling 51.5%
- 21 GPT-4.1 mini native function calling 50.5%
- 22
Qwen3-32B native function calling 48.7%
- 23
Llama-xLAM-2-8b-fc-r native function calling 46.7%
- 24
Cohere Command A native function calling 46.5%
- 25
Qwen3-8B native function calling 42.6%
- 26
Qwen3-30B-A3B-Instruct Jul 2025, native function calling 41.4%
- 27
xLAM-2-3b-fc-r native function calling 41.2%
- 28
Qwen3-14B native function calling 41%
- 29
Mistral Large 2 Nov 2024, native function calling 38.4%
- 30
Mistral Medium 3 37.7%
- 31
Llama 4 Maverick native function calling 37.3%
- 32
Mistral Small 3.2 native function calling 37.1%
- 33
Gemini 2.5 Flash-Lite Sep 2025, native function calling 36.9%
- 34 qwen3-4b-instruct-2507 native function calling 35.7%
- 35 GPT-4.1 nano native function calling 33%
- 36
Command R7B native function calling 32.1%
- 37
Llama 3.3 70B Instruct native function calling 31.9%
- 38
xLAM-2-1b-fc-r native function calling 30.4%
- 39
Gemma 3 12B prompted 30.4%
- 40
Gemma 3 27B prompted 29.5%
- 41
Phi-4 prompted 28.8%
- 42
Qwen3-1.7B native function calling 28.4%
- 43
Llama 4 Scout native function calling 28.1%
- 44 Palmyra X4 native function calling 27.9%
- 45
Mistral NeMo native function calling 27.6%
- 46 Granite-3.1-8B-Instruct native function calling 27.1%
- 47
Nova 2 Lite native function calling 27.1%
- 48
Falcon3-10B-Instruct native function calling 27%
- 49 Granite 3.2 8B Instruct native function calling 26.9%
- 50
Llama 3.1-8B prompted 25.8%
- 51
MiniCPM3-4B native function calling 25.6%
- 52
Amazon Nova Pro native function calling 25%
- 53
Falcon3-7B-Instruct native function calling 24%
- 54
Qwen3-0.6B native function calling 23.9%
- 55 Granite-20B-FunctionCalling native function calling 23.2%
- 56
Amazon Nova Micro native function calling 22.3%
- 57
Llama 3.2 3B native function calling 21.9%
- 58
Gemma 3 4B prompted 19.6%
- 59 granite-4.0-350m native function calling 19%
- 60
Falcon3-3B-Instruct native function calling 16.3%
- 61
Ministral 8B native function calling 11.1%
- 62
Falcon3-1B-Instruct native function calling 11.1%
- 63
Llama 3.2 1B native function calling 10.8%
- 64
Llama-3.1-Nemotron-Ultra-253B-v1 native function calling 10%
- 65
Gemma 3 1B prompted 7.17%
Source: Official BFCL V4 leaderboard
- 77.5 Claude Opus 4.5
- 75.0
Qwen3.7-Max
- 73.4
Seed2.0 Pro
- 73.2 Claude Sonnet 4.5
- 72.9
Qwen3.5 397B-A17B
- 72.9
Qwen3.7-Plus
- 72.5
Gemini 3 Pro
- 72.4 GLM-4.6
- 72.2
Qwen3.5-122B-A10B
- 71.8
Spark-X2.6-4B
- 71.4 LFM3-1.7B
- 70.6
Ling-2.6-1T
- 70.1
Trinity Large Thinking
- 69.6 Grok 4.1 Fast
- 68.7 Claude Haiku 4.5
- 68.5
Qwen3.5-27B
- 67.7
Qwen3-Max-2026-01-23
- 67.3
Qwen3.5-35B-A3B
- 66.5
ERNIE 5.0
- 66.1
Qwen3.5-9B
- 65.7
Keye-VL-2.0-30B-A3B
- 65.0 LFM2.5-24B-A2B
- 63.3
Qwen3.5-Omni-Plus
- 63.0 o3
- 63.0 Grok 4
- 61.6
Nova 2.0 Pro Preview
- 61.4 Granite 4.2 30B
- 60.3
Nova 2 Lite
- 60.1
MiniCPM-V-4.7-35B-A3B
- 59.1 Kimi K2
- 58.3
Nova 2 Omni Preview
- 57.2
Seed1.8
- 57.1
Command A Reasoning
- 56.9 LFM2.5-2.6B
- 56.7
DeepSeek-V3.2-Exp
- 56.2
Gemini 2.5 Flash
- 55.9 GPT-5.2
- 55.5 GPT-5 mini
- 54.7
xLAM-2-32b-fc-r
- 54.0 GPT-4.1
- 53.8
Nemotron 3 Nano 30B-A3B
- 53.2 o4-mini
- 53.1
Llama-xLAM-2-70b-fc-r
- 52.9
Nemotron-Cascade 2 30B-A3B
- 52.1
Qwen3-235B-A22B-Instruct
- 51.5 GPT-5 nano
- 50.5 GPT-4.1 mini
- 50.3
Qwen3.5-4B
- 48.7
Qwen3-32B
- 48.5 LFM2.5-8B-A1B
- 46.7
Llama-xLAM-2-8b-fc-r
- 46.5
Cohere Command A
- 43.6
Qwen3.5-2B
- 42.6
Qwen3-8B
- 41.4
Qwen3-30B-A3B-Instruct
- 41.2
xLAM-2-3b-fc-r
- 41.0
Qwen3-14B
- 38.4
Mistral Large 2
- 37.7
Mistral Medium 3
- 37.3
Llama 4 Maverick
- 37.1
Mistral Small 3.2
- 36.9
Gemini 2.5 Flash-Lite
- 35.7 qwen3-4b-instruct-2507
- 33.0 GPT-4.1 nano
- 32.5 LFM2.5-VL-3B
- 32.1
Command R7B
- 31.9
Llama 3.3 70B Instruct
- 30.4
xLAM-2-1b-fc-r
- 30.4
Gemma 3 12B
- 29.5
Gemma 3 27B
- 28.8
Phi-4
- 28.4
Qwen3-1.7B
- 28.1
Llama 4 Scout
- 27.9 Palmyra X4
- 27.6
Mistral NeMo
- 27.1 Granite-3.1-8B-Instruct
- 27.0
Falcon3-10B-Instruct
- 26.9 Granite 3.2 8B Instruct
- 25.8
Llama 3.1-8B
- 25.6
MiniCPM3-4B
- 25.3
Qwen3.5-0.8B
- 25.0
Amazon Nova Pro
- 24.0
Falcon3-7B-Instruct
- 23.9
Qwen3-0.6B
- 23.2 Granite-20B-FunctionCalling
- 22.3
Amazon Nova Micro
- 21.9
Llama 3.2 3B
- 19.6
Gemma 3 4B
- 19.0 granite-4.0-350m
- 16.3
Falcon3-3B-Instruct
- 11.1
Ministral 8B
- 11.1
Falcon3-1B-Instruct
- 10.8
Llama 3.2 1B
- 10.0
Llama-3.1-Nemotron-Ultra-253B-v1
- 7.17
Gemma 3 1B
- 1 Claude Opus 4.5 native function calling 77.5%
- 2
Qwen3.7-Max 75%
- 3
Seed2.0 Pro 73.4%
- 4 Claude Sonnet 4.5 native function calling 73.2%
- 5
Qwen3.5 397B-A17B 72.9%
- 6
Qwen3.7-Plus 72.9%
- 7
Gemini 3 Pro prompted 72.5%
- 8 GLM-4.6 thinking; native function calling 72.4%
- 9
Qwen3.5-122B-A10B 72.2%
- 10
Spark-X2.6-4B 71.8%
- 11 LFM3-1.7B 71.4%
- 12
Ling-2.6-1T 70.6%
- 13
Trinity Large Thinking 70.1%
- 14 Grok 4.1 Fast reasoning; native function calling 69.6%
- 15 Claude Haiku 4.5 native function calling 68.7%
- 16
Qwen3.5-27B 68.5%
- 17
Qwen3-Max-2026-01-23 67.7%
- 18
Qwen3.5-35B-A3B 67.3%
- 19
ERNIE 5.0 66.5%
- 20
Qwen3.5-9B 66.1%
- 21
Keye-VL-2.0-30B-A3B 65.7%
- 22 LFM2.5-24B-A2B 65%
- 23
Qwen3.5-Omni-Plus 63.3%
- 24 o3 prompted 63%
- 25 Grok 4 prompted 63%
- 26
Nova 2.0 Pro Preview 61.6%
- 27 Granite 4.2 30B 61.4%
- 28
Nova 2 Lite 60.3%
- 29
MiniCPM-V-4.7-35B-A3B 60.1%
- 30 Kimi K2 Jul 2025, native function calling 59.1%
- 31
Nova 2 Omni Preview 58.3%
- 32
Seed1.8 57.2%
- 33
Command A Reasoning native function calling 57.1%
- 34 LFM2.5-2.6B 56.9%
- 35
DeepSeek-V3.2-Exp thinking; prompted 56.7%
- 36
Gemini 2.5 Flash Sep 2025, native function calling 56.2%
- 37 GPT-5.2 native function calling 55.9%
- 38 GPT-5 mini native function calling 55.5%
- 39
xLAM-2-32b-fc-r native function calling 54.7%
- 40 GPT-4.1 native function calling 54%
- 41
Nemotron 3 Nano 30B-A3B 53.8%
- 42 o4-mini native function calling 53.2%
- 43
Llama-xLAM-2-70b-fc-r native function calling 53.1%
- 44
Nemotron-Cascade 2 30B-A3B 52.9%
- 45
Qwen3-235B-A22B-Instruct Jul 2025, prompted 52.1%
- 46 GPT-5 nano native function calling 51.5%
- 47 GPT-4.1 mini native function calling 50.5%
- 48
Qwen3.5-4B 50.3%
- 49
Qwen3-32B native function calling 48.7%
- 50 LFM2.5-8B-A1B 48.5%
- 51
Llama-xLAM-2-8b-fc-r native function calling 46.7%
- 52
Cohere Command A native function calling 46.5%
- 53
Qwen3.5-2B 43.6%
- 54
Qwen3-8B native function calling 42.6%
- 55
Qwen3-30B-A3B-Instruct Jul 2025, native function calling 41.4%
- 56
xLAM-2-3b-fc-r native function calling 41.2%
- 57
Qwen3-14B native function calling 41%
- 58
Mistral Large 2 Nov 2024, native function calling 38.4%
- 59
Mistral Medium 3 37.7%
- 60
Llama 4 Maverick native function calling 37.3%
- 61
Mistral Small 3.2 native function calling 37.1%
- 62
Gemini 2.5 Flash-Lite Sep 2025, native function calling 36.9%
- 63 qwen3-4b-instruct-2507 native function calling 35.7%
- 64 GPT-4.1 nano native function calling 33%
- 65 LFM2.5-VL-3B 32.5%
- 66
Command R7B native function calling 32.1%
- 67
Llama 3.3 70B Instruct native function calling 31.9%
- 68
xLAM-2-1b-fc-r native function calling 30.4%
- 69
Gemma 3 12B prompted 30.4%
- 70
Gemma 3 27B prompted 29.5%
- 71
Phi-4 prompted 28.8%
- 72
Qwen3-1.7B native function calling 28.4%
- 73
Llama 4 Scout native function calling 28.1%
- 74 Palmyra X4 native function calling 27.9%
- 75
Mistral NeMo native function calling 27.6%
- 76 Granite-3.1-8B-Instruct native function calling 27.1%
- 77
Falcon3-10B-Instruct native function calling 27%
- 78 Granite 3.2 8B Instruct native function calling 26.9%
- 79
Llama 3.1-8B prompted 25.8%
- 80
MiniCPM3-4B native function calling 25.6%
- 81
Qwen3.5-0.8B 25.3%
- 82
Amazon Nova Pro native function calling 25%
- 83
Falcon3-7B-Instruct native function calling 24%
- 84
Qwen3-0.6B native function calling 23.9%
- 85 Granite-20B-FunctionCalling native function calling 23.2%
- 86
Amazon Nova Micro native function calling 22.3%
- 87
Llama 3.2 3B native function calling 21.9%
- 88
Gemma 3 4B prompted 19.6%
- 89 granite-4.0-350m native function calling 19%
- 90
Falcon3-3B-Instruct native function calling 16.3%
- 91
Ministral 8B native function calling 11.1%
- 92
Falcon3-1B-Instruct native function calling 11.1%
- 93
Llama 3.2 1B native function calling 10.8%
- 94
Llama-3.1-Nemotron-Ultra-253B-v1 native function calling 10%
- 95
Gemma 3 1B prompted 7.17%
Sources: Official BFCL V4 leaderboard and labs’ model cards
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 75% | May 2026 | Lab model card | |
| 2 | | 73.4% | Feb 2026 | Lab model card | |
| 3 | | 72.9% | Feb 2026 | Lab model card | |
| 4 | | 72.9% | Jun 2026 | Lab model card | |
| 5 | | 72.2% | Feb 2026 | Lab model card | |
| 6 | | 71.8% | Oct 2026 | Lab model card | |
| 7 | LFM3-1.7B | 71.4% | Oct 2026 | Lab model card | |
| 8 | | 70.6% | Apr 2026 | Lab model card | |
| 9 | | 70.1% | Apr 2026 | Lab model card | |
| 10 | | 68.5% | Feb 2026 | Lab model card | |
| 11 | | 67.7% | Jan 2026 | Lab model card | |
| 12 | | 67.3% | Feb 2026 | Lab model card | |
| 13 | | 66.5% | Feb 2026 | Lab model card | |
| 14 | | 66.1% | Mar 2026 | Lab model card | |
| 15 | | 65.7% | May 2026 | Lab model card | |
| 16 | LFM2.5-24B-A2B | 65% | Oct 2026 | Lab model card | |
| 17 | | 63.3% | Mar 2026 | Lab model card | |
| 18 | | 61.6% | Dec 2025 | Lab model card | |
| 19 | Granite 4.2 30B | 61.4% | Aug 2026 | Lab model card | |
| 20 | | 60.3% | Dec 2025 | Lab model card | |
| 21 | | 60.1% | Oct 2026 | Lab model card | |
| 22 | | 58.3% | Dec 2025 | Lab model card | |
| 23 | | 57.2% | Dec 2025 | Lab model card | |
| 24 | LFM2.5-2.6B | 56.9% | Aug 2026 | Lab model card | |
| 25 | | 53.8% | Dec 2025 | Lab model card | |
| 26 | | 52.9% | Mar 2026 | Lab model card | |
| 27 | | 50.3% | Mar 2026 | Lab model card | |
| 28 | LFM2.5-8B-A1B | 48.5% | May 2026 | Lab model card | |
| 29 | | 43.6% | Mar 2026 | Lab model card | |
| 30 | LFM2.5-VL-3B | 32.5% | Aug 2026 | Lab model card | |
| 31 | | 25.3% | Mar 2026 | Lab model card |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | 77.5% | native function calling | Nov 2025 | Official leaderboard |
| 2 | Claude Sonnet 4.5 | 73.2% | native function calling | Sep 2025 | Official leaderboard |
| 3 | 72.5% | prompted | Nov 2025 | Official leaderboard | |
| 4 | GLM-4.6 | 72.4% | thinking; native function calling | Sep 2025 | Official leaderboard |
| 5 | Grok 4.1 Fast | 69.6% | reasoning; native function calling | Nov 2025 | Official leaderboard |
| 6 | Claude Haiku 4.5 | 68.7% | native function calling | Oct 2025 | Official leaderboard |
| 7 | o3 | 63% | prompted | Apr 2025 | Official leaderboard |
| 8 | Grok 4 | 63% | prompted | Jul 2025 | Official leaderboard |
| 9 | Kimi K2 Jul 2025 | 59.1% | native function calling | Jul 2025 | Official leaderboard |
| 10 | 57.1% | native function calling | Aug 2025 | Official leaderboard | |
| 11 | 56.7% | thinking; prompted | Sep 2025 | Official leaderboard | |
| 12 | 56.2% | native function calling | Sep 2025 | Official leaderboard | |
| 13 | GPT-5.2 | 55.9% | native function calling | Dec 2025 | Official leaderboard |
| 14 | GPT-5 mini | 55.5% | native function calling | Aug 2025 | Official leaderboard |
| 15 | 54.7% | native function calling | Mar 2025 | Official leaderboard | |
| 16 | GPT-4.1 | 54% | native function calling | Apr 2025 | Official leaderboard |
| 17 | o4-mini | 53.2% | native function calling | Apr 2025 | Official leaderboard |
| 18 | 53.1% | native function calling | Mar 2025 | Official leaderboard | |
| 19 | 52.1% | prompted | Jul 2025 | Official leaderboard | |
| 20 | GPT-5 nano | 51.5% | native function calling | Aug 2025 | Official leaderboard |
| 21 | GPT-4.1 mini | 50.5% | native function calling | Apr 2025 | Official leaderboard |
| 22 | 48.7% | native function calling | Apr 2025 | Official leaderboard | |
| 23 | 46.7% | native function calling | Mar 2025 | Official leaderboard | |
| 24 | 46.5% | native function calling | Mar 2025 | Official leaderboard | |
| 25 | 42.6% | native function calling | Apr 2025 | Official leaderboard | |
| 26 | 41.4% | native function calling | Jul 2025 | Official leaderboard | |
| 27 | 41.2% | native function calling | Mar 2025 | Official leaderboard | |
| 28 | 41% | native function calling | Apr 2025 | Official leaderboard | |
| 29 | 38.4% | native function calling | Nov 2024 | Official leaderboard | |
| 30 | 37.7% | May 2025 | Official leaderboard | ||
| 31 | 37.3% | native function calling | Apr 2025 | Official leaderboard | |
| 32 | 37.1% | native function calling | Jun 2025 | Official leaderboard | |
| 33 | 36.9% | native function calling | Sep 2025 | Official leaderboard | |
| 34 | qwen3-4b-instruct-2507 | 35.7% | native function calling | Aug 2025 | Official leaderboard |
| 35 | GPT-4.1 nano | 33% | native function calling | Apr 2025 | Official leaderboard |
| 36 | 32.1% | native function calling | Dec 2024 | Official leaderboard | |
| 37 | 31.9% | native function calling | Dec 2024 | Official leaderboard | |
| 38 | 30.4% | native function calling | Mar 2025 | Official leaderboard | |
| 39 | 30.4% | prompted | Mar 2025 | Official leaderboard | |
| 40 | 29.5% | prompted | Mar 2025 | Official leaderboard | |
| 41 | 28.8% | prompted | Dec 2024 | Official leaderboard | |
| 42 | 28.4% | native function calling | Apr 2025 | Official leaderboard | |
| 43 | 28.1% | native function calling | Apr 2025 | Official leaderboard | |
| 44 | Palmyra X4 | 27.9% | native function calling | Oct 2024 | Official leaderboard |
| 45 | 27.6% | native function calling | Jul 2024 | Official leaderboard | |
| 46 | Granite-3.1-8B-Instruct | 27.1% | native function calling | Dec 2024 | Official leaderboard |
| 47 | 27.1% | native function calling | Dec 2025 | Official leaderboard | |
| 48 | 27% | native function calling | Dec 2024 | Official leaderboard | |
| 49 | Granite 3.2 8B Instruct | 26.9% | native function calling | Feb 2025 | Official leaderboard |
| 50 | 25.8% | prompted | Jul 2024 | Official leaderboard | |
| 51 | 25.6% | native function calling | Sep 2024 | Official leaderboard | |
| 52 | 25% | native function calling | Dec 2024 | Official leaderboard | |
| 53 | 24% | native function calling | Dec 2024 | Official leaderboard | |
| 54 | 23.9% | native function calling | Apr 2025 | Official leaderboard | |
| 55 | Granite-20B-FunctionCalling | 23.2% | native function calling | Jul 2024 | Official leaderboard |
| 56 | 22.3% | native function calling | Dec 2024 | Official leaderboard | |
| 57 | 21.9% | native function calling | Sep 2024 | Official leaderboard | |
| 58 | 19.6% | prompted | Mar 2025 | Official leaderboard | |
| 59 | granite-4.0-350m | 19% | native function calling | Oct 2025 | Official leaderboard |
| 60 | 16.3% | native function calling | Dec 2024 | Official leaderboard | |
| 61 | 11.1% | native function calling | Oct 2024 | Official leaderboard | |
| 62 | 11.1% | native function calling | Dec 2024 | Official leaderboard | |
| 63 | 10.8% | native function calling | Sep 2024 | Official leaderboard | |
| 64 | 10% | native function calling | Apr 2025 | Official leaderboard | |
| 65 | 7.17% | prompted | Mar 2025 | Official leaderboard |
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Opus 4.5 | 77.5% | native function calling | Nov 2025 | Official leaderboard |
| 2 | 75% | May 2026 | Lab model card | ||
| 3 | 73.4% | Feb 2026 | Lab model card | ||
| 4 | Claude Sonnet 4.5 | 73.2% | native function calling | Sep 2025 | Official leaderboard |
| 5 | 72.9% | Feb 2026 | Lab model card | ||
| 6 | 72.9% | Jun 2026 | Lab model card | ||
| 7 | 72.5% | prompted | Nov 2025 | Official leaderboard | |
| 8 | GLM-4.6 | 72.4% | thinking; native function calling | Sep 2025 | Official leaderboard |
| 9 | 72.2% | Feb 2026 | Lab model card | ||
| 10 | 71.8% | Oct 2026 | Lab model card | ||
| 11 | LFM3-1.7B | 71.4% | Oct 2026 | Lab model card | |
| 12 | 70.6% | Apr 2026 | Lab model card | ||
| 13 | 70.1% | Apr 2026 | Lab model card | ||
| 14 | Grok 4.1 Fast | 69.6% | reasoning; native function calling | Nov 2025 | Official leaderboard |
| 15 | Claude Haiku 4.5 | 68.7% | native function calling | Oct 2025 | Official leaderboard |
| 16 | 68.5% | Feb 2026 | Lab model card | ||
| 17 | 67.7% | Jan 2026 | Lab model card | ||
| 18 | 67.3% | Feb 2026 | Lab model card | ||
| 19 | 66.5% | Feb 2026 | Lab model card | ||
| 20 | 66.1% | Mar 2026 | Lab model card | ||
| 21 | 65.7% | May 2026 | Lab model card | ||
| 22 | LFM2.5-24B-A2B | 65% | Oct 2026 | Lab model card | |
| 23 | 63.3% | Mar 2026 | Lab model card | ||
| 24 | o3 | 63% | prompted | Apr 2025 | Official leaderboard |
| 25 | Grok 4 | 63% | prompted | Jul 2025 | Official leaderboard |
| 26 | 61.6% | Dec 2025 | Lab model card | ||
| 27 | Granite 4.2 30B | 61.4% | Aug 2026 | Lab model card | |
| 28 | 60.3% | Dec 2025 | Lab model card | ||
| 29 | 60.1% | Oct 2026 | Lab model card | ||
| 30 | Kimi K2 Jul 2025 | 59.1% | native function calling | Jul 2025 | Official leaderboard |
| 31 | 58.3% | Dec 2025 | Lab model card | ||
| 32 | 57.2% | Dec 2025 | Lab model card | ||
| 33 | 57.1% | native function calling | Aug 2025 | Official leaderboard | |
| 34 | LFM2.5-2.6B | 56.9% | Aug 2026 | Lab model card | |
| 35 | 56.7% | thinking; prompted | Sep 2025 | Official leaderboard | |
| 36 | 56.2% | native function calling | Sep 2025 | Official leaderboard | |
| 37 | GPT-5.2 | 55.9% | native function calling | Dec 2025 | Official leaderboard |
| 38 | GPT-5 mini | 55.5% | native function calling | Aug 2025 | Official leaderboard |
| 39 | 54.7% | native function calling | Mar 2025 | Official leaderboard | |
| 40 | GPT-4.1 | 54% | native function calling | Apr 2025 | Official leaderboard |
| 41 | 53.8% | Dec 2025 | Lab model card | ||
| 42 | o4-mini | 53.2% | native function calling | Apr 2025 | Official leaderboard |
| 43 | 53.1% | native function calling | Mar 2025 | Official leaderboard | |
| 44 | 52.9% | Mar 2026 | Lab model card | ||
| 45 | 52.1% | prompted | Jul 2025 | Official leaderboard | |
| 46 | GPT-5 nano | 51.5% | native function calling | Aug 2025 | Official leaderboard |
| 47 | GPT-4.1 mini | 50.5% | native function calling | Apr 2025 | Official leaderboard |
| 48 | 50.3% | Mar 2026 | Lab model card | ||
| 49 | 48.7% | native function calling | Apr 2025 | Official leaderboard | |
| 50 | LFM2.5-8B-A1B | 48.5% | May 2026 | Lab model card | |
| 51 | 46.7% | native function calling | Mar 2025 | Official leaderboard | |
| 52 | 46.5% | native function calling | Mar 2025 | Official leaderboard | |
| 53 | 43.6% | Mar 2026 | Lab model card | ||
| 54 | 42.6% | native function calling | Apr 2025 | Official leaderboard | |
| 55 | 41.4% | native function calling | Jul 2025 | Official leaderboard | |
| 56 | 41.2% | native function calling | Mar 2025 | Official leaderboard | |
| 57 | 41% | native function calling | Apr 2025 | Official leaderboard | |
| 58 | 38.4% | native function calling | Nov 2024 | Official leaderboard | |
| 59 | 37.7% | May 2025 | Official leaderboard | ||
| 60 | 37.3% | native function calling | Apr 2025 | Official leaderboard | |
| 61 | 37.1% | native function calling | Jun 2025 | Official leaderboard | |
| 62 | 36.9% | native function calling | Sep 2025 | Official leaderboard | |
| 63 | qwen3-4b-instruct-2507 | 35.7% | native function calling | Aug 2025 | Official leaderboard |
| 64 | GPT-4.1 nano | 33% | native function calling | Apr 2025 | Official leaderboard |
| 65 | LFM2.5-VL-3B | 32.5% | Aug 2026 | Lab model card | |
| 66 | 32.1% | native function calling | Dec 2024 | Official leaderboard | |
| 67 | 31.9% | native function calling | Dec 2024 | Official leaderboard | |
| 68 | 30.4% | native function calling | Mar 2025 | Official leaderboard | |
| 69 | 30.4% | prompted | Mar 2025 | Official leaderboard | |
| 70 | 29.5% | prompted | Mar 2025 | Official leaderboard | |
| 71 | 28.8% | prompted | Dec 2024 | Official leaderboard | |
| 72 | 28.4% | native function calling | Apr 2025 | Official leaderboard | |
| 73 | 28.1% | native function calling | Apr 2025 | Official leaderboard | |
| 74 | Palmyra X4 | 27.9% | native function calling | Oct 2024 | Official leaderboard |
| 75 | 27.6% | native function calling | Jul 2024 | Official leaderboard | |
| 76 | Granite-3.1-8B-Instruct | 27.1% | native function calling | Dec 2024 | Official leaderboard |
| 77 | 27% | native function calling | Dec 2024 | Official leaderboard | |
| 78 | Granite 3.2 8B Instruct | 26.9% | native function calling | Feb 2025 | Official leaderboard |
| 79 | 25.8% | prompted | Jul 2024 | Official leaderboard | |
| 80 | 25.6% | native function calling | Sep 2024 | Official leaderboard | |
| 81 | 25.3% | Mar 2026 | Lab model card | ||
| 82 | 25% | native function calling | Dec 2024 | Official leaderboard | |
| 83 | 24% | native function calling | Dec 2024 | Official leaderboard | |
| 84 | 23.9% | native function calling | Apr 2025 | Official leaderboard | |
| 85 | Granite-20B-FunctionCalling | 23.2% | native function calling | Jul 2024 | Official leaderboard |
| 86 | 22.3% | native function calling | Dec 2024 | Official leaderboard | |
| 87 | 21.9% | native function calling | Sep 2024 | Official leaderboard | |
| 88 | 19.6% | prompted | Mar 2025 | Official leaderboard | |
| 89 | granite-4.0-350m | 19% | native function calling | Oct 2025 | Official leaderboard |
| 90 | 16.3% | native function calling | Dec 2024 | Official leaderboard | |
| 91 | 11.1% | native function calling | Oct 2024 | Official leaderboard | |
| 92 | 11.1% | native function calling | Dec 2024 | Official leaderboard | |
| 93 | 10.8% | native function calling | Sep 2024 | Official leaderboard | |
| 94 | 10% | native function calling | Apr 2025 | Official leaderboard | |
| 95 | 7.17% | prompted | Mar 2025 | Official leaderboard |
The benchmark's own leaderboard has 80 entries, 65 of them models we track. Switch the scores above to rank them together.
Rankings by part
We read 24 parts from the benchmark’s own leaderboard in 6 groups: non-live, live, multi turn, web search, memory and detection. Each ranks models on one part of the test. Claude Opus 4.5 leads 6 of them, and 10 other models lead the rest. Arrows show how far a model moved from its overall rank among the models we track.
Non-Live AST Acc
65 models we track have a score here.
- 1
Gemini 3 Pro ↑2 90.7%
- 2
Qwen3-235B-A22B-Instruct Jul 2025 ↑17 90.3%
- 3
Qwen3-32B ↑19 90.3%
- 4
Mistral Small 3.2 ↑28 89.7%
- 5 Claude Opus 4.5 ↓4 89.7%
- 6
xLAM-2-32b-fc-r ↑9 89.6%
- 7
Qwen3-14B ↑21 89.5%
- 8
Llama 4 Scout ↑35 89.4%
- 9
Qwen3-30B-A3B-Instruct Jul 2025 ↑17 88.9%
- 10 GPT-4.1 ↑6 88.7%
Non-Live Simple AST
65 models we track have a score here.
- 1
Qwen3-30B-A3B-Instruct Jul 2025 ↑25 80.7%
- 2
xLAM-2-32b-fc-r ↑13 80.4%
- 3
Qwen3-235B-A22B-Instruct Jul 2025 ↑16 79.8%
- 4
Gemini 3 Pro ↓1 79.6%
- 5 Claude Opus 4.5 ↓4 79.6%
- 6
Mistral NeMo ↑39 79.3%
- 7
Qwen3-32B ↑15 79.1%
- 8
Llama 4 Scout ↑35 79%
- 9
Mistral Small 3.2 ↑23 78.8%
- 10
Llama-xLAM-2-70b-fc-r ↑8 78.3%
Non-Live Multiple AST
65 models we track have a score here.
- 1
Qwen3-32B ↑21 97%
- 2
Qwen3-8B ↑23 96.5%
- 3 Palmyra X4 ↑41 96%
- 4
Mistral Small 3.2 ↑28 96%
- 5
Gemini 3 Pro ↓2 96%
- 6 Claude Sonnet 4.5 ↓4 95.5%
- 7 Claude Opus 4.5 ↓6 95.5%
- 8
Llama 3.1-8B ↑42 95%
- 9
Llama 3.3 70B Instruct ↑28 95%
- 10
Llama 4 Maverick ↑21 95%
Non-Live Parallel AST
65 models we track have a score here.
- 1
Gemini 2.5 Flash Sep 2025 ↑11 96%
- 2
Qwen3-14B ↑26 95.5%
- 3
Qwen3-235B-A22B-Instruct Jul 2025 ↑16 95.5%
- 4
Gemini 3 Pro ↓1 95%
- 5
Qwen3-8B ↑20 94.5%
- 6 Claude Sonnet 4.5 ↓4 94.5%
- 7
Llama 4 Maverick ↑24 94%
- 8
Llama 4 Scout ↑35 94%
- 9 GPT-4.1 ↑7 94%
- 10
Qwen3-30B-A3B-Instruct Jul 2025 ↑16 94%
Non-Live Parallel Multiple AST
65 models we track have a score here.
- 1 Claude Opus 4.5 92.5%
- 2
Qwen3-14B ↑26 92%
- 3
Gemini 2.5 Flash-Lite Sep 2025 ↑30 92%
- 4 Claude Sonnet 4.5 ↓2 92%
- 5
Gemini 3 Pro ↓2 92%
- 6
Mistral NeMo ↑39 91.5%
- 7
Qwen3-32B ↑15 91.5%
- 8
Mistral Small 3.2 ↑24 91.5%
- 9
Llama 3.3 70B Instruct ↑28 91%
- 10
xLAM-2-32b-fc-r ↑5 91%
Live Acc
65 models we track have a score here.
- 1
Gemini 3 Pro ↑2 83.1%
- 2
Qwen3-32B ↑20 82%
- 3
Mistral Large 2 Nov 2024 ↑26 81.9%
- 4 Claude Sonnet 4.5 ↓2 81.1%
- 5 GLM-4.6 ↓1 80.9%
- 6
Nova 2 Lite ↑41 80.8%
- 7
Qwen3-8B ↑18 80.5%
- 8
Qwen3-14B ↑20 80%
- 9 Claude Opus 4.5 ↓8 79.8%
- 10
Mistral Small 3.2 ↑22 79%
Live Simple AST
65 models we track have a score here.
- 1
Qwen3-32B ↑21 89.5%
- 2 Claude Sonnet 4.5 89.5%
- 3 GLM-4.6 ↑1 89.5%
- 4
Gemini 3 Pro ↓1 87.6%
- 5
Mistral Large 2 Nov 2024 ↑24 87.2%
- 6
Gemini 2.5 Flash Sep 2025 ↑6 87.2%
- 7 Claude Opus 4.5 ↓6 86.4%
- 8
Gemma 3 12B ↑31 85.7%
- 9
Cohere Command A ↑15 85.7%
- 10
Qwen3-14B ↑18 85.7%
Live Multiple AST
65 models we track have a score here.
- 1
Gemini 3 Pro ↑2 81.8%
- 2
Qwen3-32B ↑20 81.2%
- 3
Mistral Large 2 Nov 2024 ↑26 80.7%
- 4
Nova 2 Lite ↑43 80.2%
- 5
Qwen3-8B ↑20 79.7%
- 6
Mistral Small 3.2 ↑26 79.4%
- 7
Qwen3-14B ↑21 79%
- 8 Claude Sonnet 4.5 ↓6 78.9%
- 9 GLM-4.6 ↓5 78.9%
- 10
Command A Reasoning 78.3%
Live Parallel AST
65 models we track have a score here.
- 1
Mistral Large 2 Nov 2024 ↑28 93.8%
- 2
Gemma 3 27B ↑38 93.8%
- 3
Qwen3-8B ↑22 93.8%
- 4
Mistral Small 3.2 ↑28 93.8%
- 5
Gemini 3 Pro ↓2 93.8%
- 6
Mistral NeMo ↑39 87.5%
- 7
Gemma 3 12B ↑32 87.5%
- 8 GPT-4.1 ↑8 87.5%
- 9 GPT-4.1 nano ↑26 87.5%
- 10
Qwen3-14B ↑18 87.5%
Live Parallel Multiple AST
65 models we track have a score here.
- 1
Gemini 3 Pro ↑2 87.5%
- 2 Claude Sonnet 4.5 83.3%
- 3
Llama 4 Scout ↑40 79.2%
- 4
Qwen3-8B ↑21 79.2%
- 5 Grok 4 ↑3 79.2%
- 6
Nova 2 Lite ↑41 79.2%
- 7
Mistral Large 2 Nov 2024 ↑22 75%
- 8 GPT-4.1 ↑8 75%
- 9
Qwen3-14B ↑19 75%
- 10
Qwen3-1.7B ↑32 75%
Multi Turn Acc
65 models we track have a score here.
- 1
Llama-xLAM-2-70b-fc-r ↑17 77.4%
- 2
Llama-xLAM-2-8b-fc-r ↑21 70%
- 3
xLAM-2-32b-fc-r ↑12 69.5%
- 4 Claude Opus 4.5 ↓3 68.4%
- 5 GLM-4.6 ↓1 68%
- 6
Gemini 3 Pro ↓3 63.1%
- 7 o3 62.3%
- 8 Claude Sonnet 4.5 ↓6 61.4%
- 9 Grok 4.1 Fast ↓4 58.9%
- 10
xLAM-2-3b-fc-r ↑17 58.4%
Multi Turn Base
65 models we track have a score here.
- 1
Llama-xLAM-2-70b-fc-r ↑17 82.5%
- 2
xLAM-2-32b-fc-r ↑13 81.5%
- 3 Claude Opus 4.5 ↓2 81%
- 4
Llama-xLAM-2-8b-fc-r ↑19 76%
- 5 GLM-4.6 ↓1 74.5%
- 6
xLAM-2-3b-fc-r ↑21 71.5%
- 7 Grok 4.1 Fast ↓2 70.5%
- 8 Claude Sonnet 4.5 ↓6 69%
- 9
Gemini 3 Pro ↓6 69%
- 10 o3 ↓3 68%
Multi Turn Miss Func
65 models we track have a score here.
- 1
Llama-xLAM-2-70b-fc-r ↑17 77%
- 2
xLAM-2-32b-fc-r ↑13 72.5%
- 3
Llama-xLAM-2-8b-fc-r ↑20 72%
- 4 GLM-4.6 68%
- 5 Claude Sonnet 4.5 ↓3 65%
- 6 Claude Opus 4.5 ↓5 64%
- 7 o3 63.5%
- 8
Gemini 3 Pro ↓5 63%
- 9 Grok 4.1 Fast ↓4 59.5%
- 10
xLAM-2-3b-fc-r ↑17 59%
Multi Turn Miss Param
65 models we track have a score here.
- 1
Llama-xLAM-2-70b-fc-r ↑17 74%
- 2
xLAM-2-32b-fc-r ↑13 67.5%
- 3
Llama-xLAM-2-8b-fc-r ↑20 65%
- 4 GLM-4.6 63%
- 5 Claude Opus 4.5 ↓4 58%
- 6
xLAM-2-3b-fc-r ↑21 57.5%
- 7
Gemini 3 Pro ↓4 56.5%
- 8 o3 ↓1 54.5%
- 9 Claude Sonnet 4.5 ↓7 52.5%
- 10 Claude Haiku 4.5 ↓4 52.5%
Multi Turn Long Context
65 models we track have a score here.
- 1
Llama-xLAM-2-70b-fc-r ↑17 76%
- 2 Claude Opus 4.5 ↓1 70.5%
- 3
Llama-xLAM-2-8b-fc-r ↑20 67%
- 4 GLM-4.6 66.5%
- 5
Gemini 3 Pro ↓2 64%
- 6 o3 ↑1 63%
- 7 Grok 4.1 Fast ↓2 62.5%
- 8 Claude Sonnet 4.5 ↓6 59%
- 9
xLAM-2-32b-fc-r ↑6 56.5%
- 10 Claude Haiku 4.5 ↓4 56%
Web Search Acc
65 models we track have a score here.
- 1 Claude Opus 4.5 84.5%
- 2 Claude Haiku 4.5 ↑4 83.5%
- 3 Grok 4.1 Fast ↑2 82.5%
- 4 Grok 4 ↑4 82%
- 5 GPT-5 mini ↑9 82%
- 6 Claude Sonnet 4.5 ↓4 81%
- 7
Gemini 3 Pro ↓4 80%
- 8 GLM-4.6 ↓4 77.5%
- 9 o3 ↓2 77%
- 10 o4-mini ↑7 75.5%
Web Search Base
65 models we track have a score here.
- 1 GPT-5 mini ↑13 87%
- 2 Claude Haiku 4.5 ↑4 86%
- 3 Claude Opus 4.5 ↓2 84%
- 4 Claude Sonnet 4.5 ↓2 82%
- 5 Grok 4.1 Fast 82%
- 6 Grok 4 ↑2 80%
- 7
DeepSeek-V3.2-Exp ↑4 80%
- 8 o3 ↓1 79%
- 9 GLM-4.6 ↓5 79%
- 10
Gemini 3 Pro ↓7 78%
Web Search No Snippet
65 models we track have a score here.
- 1 Claude Opus 4.5 85%
- 2 Grok 4 ↑6 84%
- 3 Grok 4.1 Fast ↑2 83%
- 4
Gemini 3 Pro ↓1 82%
- 5 Claude Haiku 4.5 ↑1 81%
- 6 Claude Sonnet 4.5 ↓4 80%
- 7 GPT-5 mini ↑7 77%
- 8 o4-mini ↑9 76%
- 9 GLM-4.6 ↓5 76%
- 10 o3 ↓3 75%
Memory Acc
65 models we track have a score here.
- 1 Claude Opus 4.5 73.8%
- 2 Claude Sonnet 4.5 65%
- 3
Gemini 3 Pro 61.7%
- 4 Grok 4 ↑4 55.9%
- 5 GLM-4.6 ↓1 55.7%
- 6 Claude Haiku 4.5 54.4%
- 7
DeepSeek-V3.2-Exp ↑4 54.2%
- 8 Grok 4.1 Fast ↓3 54%
- 9 o3 ↓2 51.8%
- 10 GPT-5.2 ↑3 45.8%
Memory KV
65 models we track have a score here.
- 1 Claude Opus 4.5 71%
- 2
Gemini 3 Pro ↑1 59.4%
- 3 Grok 4 ↑5 57.4%
- 4 Claude Sonnet 4.5 ↓2 54.2%
- 5 Claude Haiku 4.5 ↑1 51.6%
- 6
DeepSeek-V3.2-Exp ↑5 46.5%
- 7 GLM-4.6 ↓3 43.9%
- 8 Grok 4.1 Fast ↓3 41.3%
- 9 GPT-5 mini ↑5 36.8%
- 10 o3 ↓3 33.5%
Memory Vector
65 models we track have a score here.
- 1 Claude Opus 4.5 72.9%
- 2
Gemini 3 Pro ↑1 63.2%
- 3
DeepSeek-V3.2-Exp ↑8 61.3%
- 4 Grok 4 ↑4 59.4%
- 5 Claude Sonnet 4.5 ↓3 57.4%
- 6 Grok 4.1 Fast ↓1 57.4%
- 7 GLM-4.6 ↓3 56.1%
- 8 Claude Haiku 4.5 ↓2 55.5%
- 9 o3 ↓2 50.3%
- 10
Gemini 2.5 Flash Sep 2025 ↑2 50.3%
Memory Recursive Summarization
65 models we track have a score here.
- 1 Claude Sonnet 4.5 ↑1 83.2%
- 2 Claude Opus 4.5 ↓1 77.4%
- 3 o3 ↑4 72.9%
- 4 GLM-4.6 67.1%
- 5
Gemini 3 Pro ↓2 63.2%
- 6 Grok 4.1 Fast ↓1 63.2%
- 7 GPT-5.2 ↑6 60.6%
- 8
DeepSeek-V3.2-Exp ↑3 59.4%
- 9 o4-mini ↑8 58.7%
- 10 Claude Haiku 4.5 ↓4 56.1%
Relevance Detection
65 models we track have a score here.
- 1
Llama 3.3 70B Instruct ↑36 100%
- 2
Falcon3-7B-Instruct ↑51 100%
- 3
Gemma 3 4B ↑55 100%
- 4
Llama 4 Maverick ↑27 100%
- 5
Llama 4 Scout ↑38 100%
- 6 GPT-4.1 ↑10 100%
- 7
Mistral NeMo ↑38 93.8%
- 8
Llama 3.1-8B ↑42 93.8%
- 9
Mistral Large 2 Nov 2024 ↑20 93.8%
- 10
Amazon Nova Pro ↑42 93.8%
Irrelevance Detection
65 models we track have a score here.
- 1
Ministral 8B ↑60 100%
- 2
Llama-3.1-Nemotron-Ultra-253B-v1 ↑62 100%
- 3 Claude Haiku 4.5 ↑3 95.3%
- 4 Claude Sonnet 4.5 ↓2 95%
- 5
Gemini 2.5 Flash Sep 2025 ↑7 93.7%
- 6
Gemini 2.5 Flash-Lite Sep 2025 ↑27 93.3%
- 7
DeepSeek-V3.2-Exp ↑4 93.2%
- 8
Mistral Medium 3 ↑22 92%
- 9 GPT-5 mini ↑5 91%
- 10 Claude Opus 4.5 ↓9 90.8%
The best score, 77.5%, costs $87 for the full run
Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
Source: Official BFCL V4 leaderboard
| Model | Effort | Harness | Score | Cost of the full run | Output tokens |
|---|---|---|---|---|---|
| Claude Opus 4.5 | native function calling | 77.5% | $87 | ||
| Claude Sonnet 4.5 | native function calling | 73.2% | $44 | ||
| prompted | 72.5% | $299 | |||
| GLM-4.6 | thinking; native function calling | 72.4% | $4.64 | ||
| Grok 4.1 Fast | reasoning; native function calling | 69.6% | $17 | ||
| Claude Haiku 4.5 | native function calling | 68.7% | $14 | ||
| native function calling | 68.1% | $225 | |||
| o3 | prompted | 63% | $235 | ||
| Grok 4 | prompted | 63% | $348 | ||
| Grok 4 | native function calling | 61.4% | $355 | ||
| Kimi K2 Jul 2025 | native function calling | 59.1% | $6.19 | ||
| Grok 4.1 Fast | non-reasoning; native function calling | 58.3% | $16 | ||
| native function calling | 57.1% | $3.04 | |||
| thinking; prompted | 56.7% | $58 | |||
| native function calling | 56.2% | $26 | |||
| GPT-5.2 | native function calling | 55.9% | $86 | ||
| GPT-5 mini | native function calling | 55.5% | $22 | ||
| native function calling | 54.7% | $6.00 | |||
| native function calling | 54.1% | $6.71 | |||
| GPT-4.1 | native function calling | 54% | $101 | ||
| o4-mini | native function calling | 53.2% | $82 | ||
| native function calling | 53.1% | $25 | |||
| prompted | 52.1% | $3.12 | |||
| GPT-5 nano | native function calling | 51.5% | $8.79 | ||
| prompted | 50.9% | $33 | |||
| GPT-4.1 mini | native function calling | 50.5% | $19 | ||
| o4-mini | prompted | 50.3% | $134 | ||
| native function calling | 48.7% | $153 | |||
| o3 | native function calling | 48.6% | $133 | ||
| native function calling | 48% | $2.50 | |||
| prompted | 46.8% | $200 | |||
| native function calling | 46.7% | $21 | |||
| native function calling | 46.5% | $91 | |||
| GPT-5.2 | prompted | 45.3% | $165 | ||
| native function calling | 42.6% | $43 | |||
| native function calling | 41.4% | $5.62 | |||
| native function calling | 41.2% | $3.36 | |||
| native function calling | 41% | $3.38 | |||
| prompted | 40.4% | $64 | |||
| GPT-4.1 | prompted | 39.4% | $146 | ||
| native function calling | 38.4% | $116 | |||
| prompted | 37.8% | $1.35 | |||
| 37.7% | $37 | ||||
| native function calling | 37.6% | $19 | |||
| native function calling | 37.3% | $18 | |||
| native function calling | 37.1% | $5.20 | |||
| native function calling | 36.9% | $7.55 | |||
| prompted | 36.7% | $1.56 | |||
| qwen3-4b-instruct-2507 | native function calling | 35.7% | $6.37 | ||
| qwen3-4b-instruct-2507 | prompted | 35.5% | $54 | ||
| Claude Opus 4.5 | prompted | 33.5% | $88 | ||
| GPT-4.1 nano | native function calling | 33% | $5.66 | ||
| prompted | 32.4% | $6.91 | |||
| native function calling | 32.1% | $1.50 | |||
| native function calling | 31.9% | $30 | |||
| prompted | 31.8% | $232 | |||
| native function calling | 30.4% | $2.79 | |||
| prompted | 30.4% | $11 | |||
| GPT-4.1 mini | prompted | 29.7% | $21 | ||
| prompted | 29.5% | $12 | |||
| prompted | 28.8% | $8.72 | |||
| native function calling | 28.4% | $4.33 | |||
| native function calling | 28.1% | $25 | |||
| prompted | 28% | $7.05 | |||
| Palmyra X4 | native function calling | 27.9% | $178 | ||
| GPT-5 mini | prompted | 27.8% | $83 | ||
| native function calling | 27.6% | $8.12 | |||
| GPT-5 nano | prompted | 27.6% | $21 | ||
| Granite-3.1-8B-Instruct | native function calling | 27.1% | $9.32 | ||
| native function calling | 27.1% | $78 | |||
| native function calling | 27% | $53 | |||
| Granite 3.2 8B Instruct | native function calling | 26.9% | $25 | ||
| prompted | 25.8% | $7.49 | |||
| native function calling | 25.6% | $54 | |||
| Claude Haiku 4.5 | prompted | 25.3% | $45 | ||
| native function calling | 25% | $48 | |||
| Claude Sonnet 4.5 | prompted | 24.9% | $48 | ||
| GPT-4.1 nano | prompted | 24.9% | $7.42 | ||
| native function calling | 24% | $74 | |||
| native function calling | 23.9% | $0.46 | |||
| Granite-20B-FunctionCalling | native function calling | 23.2% | $5.23 | ||
| prompted | 22.4% | $3.65 | |||
| native function calling | 22.3% | $1.81 | |||
| prompted | 22.1% | $30 | |||
| native function calling | 21.9% | $6.20 | |||
| prompted | 19.6% | $4.14 | |||
| prompted | 19.3% | $14 | |||
| granite-4.0-350m | native function calling | 19% | $1.44 | ||
| native function calling | 16.3% | $37 | |||
| native function calling | 11.1% | $70 | |||
| native function calling | 11.1% | $1.72 | |||
| native function calling | 10.8% | $1.64 | |||
| native function calling | 10% | $0.72 | |||
| prompted | 7.17% | $3.40 |
4 models have held number one in labs’ own numbers since its release in July 2025 3 models have held number one across all independent runs since its release in July 2025 3 models have held number one with claims included since its release in July 2025
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Sources: Official BFCL V4 leaderboard and labs’ model cards
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Sources: Official BFCL V4 leaderboard and labs’ model cards
5 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| Oct 27, 2026 | LFM3-1.7B | 71.4 | BFCL V4 |
| Oct 22, 2026 | 71.8 | BFCL-V4 | |
| Oct 13, 2026 | LFM2.5-24B-A2B | 65 | BFCLv4 |
| Oct 7, 2026 | 60.1 | BFCL-V4 | |
| Aug 25, 2026 | Granite 4.2 30B | 61.4 | BFCL (v4) |
| Aug 12, 2026 | LFM2.5-VL-3B | 32.5 | BFCL V4 |
| Aug 4, 2026 | LFM2.5-2.6B | 56.9 | BFCLv4 |
| Jun 1, 2026 | 72.9 | BFCL-V4 | |
| May 28, 2026 | LFM2.5-8B-A1B | 48.5 | BFCLv4 |
| May 25, 2026 | 65.7 | BFCL-V4 | |
| May 20, 2026 | 75 | BFCL-V4 | |
| Apr 29, 2026 | 70.6 | BFCL-V4 | |
| Apr 1, 2026 | 70.1 | BCFLv4 | |
| Mar 30, 2026 | 63.3 | BFCL-V4 | |
| Mar 19, 2026 | 52.9 | BFCL v4 | |
| Mar 2, 2026 | 25.3 | BFCL-V4 | |
| Mar 2, 2026 | 43.6 | BFCL-V4 | |
| Mar 2, 2026 | 50.3 | BFCL-V4 | |
| Mar 2, 2026 | 66.1 | BFCL-V4 | |
| Feb 24, 2026 | 67.3 | BFCL-V4 | |
| Feb 24, 2026 | 68.5 | BFCL-V4 | |
| Feb 24, 2026 | 72.2 | BFCL-V4 | |
| Feb 16, 2026 | 72.9 | BFCL-V4 | |
| Feb 14, 2026 | 73.4 | BFCL-v4 | |
| Feb 6, 2026 | 66.5 | BFCL v4 | |
| Jan 23, 2026 | 67.7 | BFCL-V4 | |
| Dec 18, 2025 | 57.2 | BFCL-v4 | |
| Dec 15, 2025 | 53.8 | BFCL v4 | |
| Dec 2, 2025 | 58.3 | BFCL v4 Overall FC | |
| Dec 2, 2025 | 60.3 | BFCL v4 Overall FC | |
| Dec 2, 2025 | 61.6 | BFCL v4 Overall FC | |
| Nov 19, 2025 | Grok 4.1 Fast | Berkeley Function Calling v4 Benchmark, overall accuracy (chart) |
11 system cards report it, from 7 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| Jun 13, 2026 | | 70.6 | BFCL-v4, accuracy (Table 5) | Capabilities |
| Jun 9, 2026 | | 65.7 | BFCL-V4 (Table 4) | Capabilities |
| Apr 17, 2026 | | 63.3 | BFCL-V4 (Table 4) | Capabilities |
| Mar 21, 2026 | | 57.2 | BFCL-v4 (Table 4) | Capabilities |
| Mar 19, 2026 | | 52.9 | BFCL v4, thinking mode, avg@1 (Table 1) | Capabilities |
| Feb 14, 2026 | | 73.4 | BFCL-v4 (Table 11) | Capabilities |
| Feb 4, 2026 | | 66.5 | BFCL v4 (Table 2) | Capabilities |
| Dec 23, 2025 | | 53.8 | BFCL v4 (Table 3) | Capabilities |
| Dec 2, 2025 | | 58.3 | BFCL v4 Overall FC (Table 5) | Capabilities |
| Dec 2, 2025 | | 60.3 | BFCL v4 Overall FC (Table 2) | Capabilities |
| Dec 2, 2025 | | 61.6 | BFCL v4 Overall FC (Table 2) | Capabilities |
One lab claim next to the leaderboard
Each score a lab reported for its own model, beside the same model's best score on the leaderboard. Gaps mostly come from different harnesses, effort levels and task subsets. Every benchmark.
| Model | Lab says | Board says | Gap |
|---|---|---|---|
| | 60.3% BFCL v4 Overall FC | 27.1% Leaderboard, native function calling | +33 pts |
3 logged changes that move scores
Changes by the maintainers, so a score from before a change may not compare with one from after it. Newest first, each with the maintainers' own source. The list may be incomplete. What we log.
- Scores recomputed December 17th, 2025
The maintainers updated leaderboard scores using checkpoint f7cf735 after backend and dataset fixes. The new scores supersede the preceding checkpoint’s results. Source
- Scores recomputed November 4th, 2025
The maintainers updated leaderboard scores using checkpoint 86d0374 after dataset, backend, and grading fixes. Earlier leaderboard scores use different evaluation behavior. Source
- Questions corrected October 13th, 2025
The maintainers corrected prompts and ground truths across single-turn and multi-turn categories. Evaluations now use revised questions and expected calls. Source
Versions
We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.
Source: Labs’ model cards
More in Tool use
- AutomationBench In use Agent completes end-to-end business workflows inside a simulated company exposing dozens of REST APIs. Graded on resulting state.
Gemini 4 Neon 56.3%
- MCP Atlas In use Multi-step workflows over real MCP servers. Scores tool-use competency on realistic tasks.
Gemini 4 Argon 90.7%
- τ³-bench Banking In use Agents act as banking customer-service reps over many turns with a simulated user, following policy. Pass^1 on the final database state.
Qwen 3.8 Max 55.1%
- Toolathlon-Verified In use 108 long-horizon tasks across 32 apps through about 600 MCP tools. Execution-based checkers inspect final environment state. GLM-5.3-Flash 78.4%
- DeepSearchQA In use Questions that need multi-step search and synthesis. Final answers graded against references. Kimi K3 95%
- Claw-Eval In use 300 human-verified agent tasks (service orchestration, multimodal, multi-turn dialogue) audited via traces and snapshots. Rubric scores with Pass^3 consistency. Claude Sonnet 4.6 81.4%
Data audit
10 of 11 checks pass, 1 warning. Look first at the Lab claims check. 1 of 1 pair differs by more than 10 points, the largest for Nova 2 Lite (+33 pts). Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 4 pass 1 note
- Pass Independent runs Scores from the official leaderboard (80 entries), launch posts (31 scores) and system cards (11 scores).
- Pass Models scored 95 models we track, plus 15 other entries on its leaderboard.
- Pass Newest score The newest is from October 2026, for LFM3-1.7B.
- Note Frontier models None of the top 10 has a score. The strongest tested is Qwen3.7-Max (#49).
- Pass Headroom The best is 77.5%, under 85%, so it still separates models.
Leaderboard import 3 pass
- Pass Importer We read the board on gorilla.cs.berkeley.edu: 94 rows for models we track and 15 other entries at the last read.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 warning 1 pass
- Warning Lab claims 1 of 1 pair differs by more than 10 points, the largest for Nova 2 Lite (+33 pts).
- Pass Known issues No published audit found problems.