Benchmark Almanac

Blobfish AI

We track 11 benchmarks that Blobfish AI made or helped make.

0 in use 7 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
FactoryBench-100 Agents complete 100 manufacturing workflows through simulated ERP, email, spreadsheet, and chat tools. FactoryScore checks their decisions and the state left in each system. Tasks from real jobs Aug 2026 76.2% Not in cards
Blobfish Litigation Bench Agents complete 100 tasks in a simulated litigation practice. The grader compares the changes agents make to case records against required outcomes. Law Aug 2026 Not in cards
CounselBench-100 Agents investigate legal matters and decide which actions the evidence supports. The verifier checks decisions, changes to case records, and advice across 100 synthetic tasks. Law Aug 2026 Not in cards
DealBench-100 Agents perform 100 investment banking workflows across ten synthetic transactions. Deterministic checks grade calculations, decisions, committed changes, and consistency between models and presentations. Finance Sep 2026 Not in cards
DevOpsBench-100 Agents resolve 100 operations and reliability tasks in a simulated online store. The verifier compares system state before and after their work. Terminal tasks Aug 2026 Not in cards
ERPBench-100 Agents execute 100 business workflows across ten simulated enterprise resource planning systems. Checks grade decisions, calculations, committed state, and handoffs on a 100-point scale. Apps and MCP servers Sep 2026 Not in cards
HubBench Agents make workplace decisions across 13 professional domains using connected software tools and evidence. Version 1.4.0 grades 104 tasks with deterministic state checks. Tasks from real jobs Sep 2026 Not in cards
LedgerBench-100 Agents perform 100 corporate finance tasks through 66 tools in a shared enterprise system. SQL checks grade the changes they make. Finance Aug 2026 Not in cards
SalesBench-100 Agents reconcile sales evidence, update authorized records, and deliver executive briefs across 100 workflows. Checks grade procedure, system state, and outputs. Apps and MCP servers Aug 2026 Not in cards
SemiKongBench-100 Agents resolve 100 synthetic semiconductor operating exceptions. The evaluator grades each workflow with deterministic checks. Tasks from real jobs Sep 2026 Not in cards
Typed Decision Bench Models assign probabilities to decisions about a given state. The benchmark scores accuracy and calibration across five suites. Running a business Sep 2026 Not in cards