Benchmark Almanac

Ramp Labs

We track 11 benchmarks and 1 arena that Ramp Labs made or helped make.

0 in use 0 with public code 9 with a leaderboard here

Arena leaderboards

Top five in each of its 1 arena with three or more rated models, the most rated first.

Benchmarks

Name Kind Released Best score Status
Ramp SWE-Bench v1 Agents modify Ramp’s backend code to satisfy engineering requests. Hidden tests check whether each patch solves the task. Fixing real repositories Jun 2026 88.5% Not in cards
Ramp SWE-Bench v2 Agents solve 500 engineering tasks from Ramp's production backend. A panel of model judges compares each patch with the code engineers merged. Fixing real repositories Nov 2026 76.4% Not in cards
Accounting Autocoding Models select a bookkeeping category for each expense. Ramp checks the first choice against the assigned category. Finance Mar 2026 92.6% Not in cards
Ramp Accounting Bench Agents reconcile accounts and prepare financial records across 22 simulated companies. A model grades their answers and files against accounting rubrics. Finance Nov 2026 61.1% Not in cards
Fund Smart Routing Agent Models choose a fund for a transaction. Ramp measures how often a user overrides that choice. Finance Mar 2026 100% Not in cards
Policy Agent Models approve expenses, reject them, or request human review. Ramp compares their decisions with human expense reviews. Finance Mar 2026 100% Not in cards
Agentic Credit Analysis Models extract and standardize income statements and balance sheets. Ramp compares their figures with credit analysts’ figures using a five percent tolerance. Finance Oct 2026 80.2% Not in cards
Ramp Stack Accounting Benchmark Agents perform month-end accounting work across eight simulated businesses. Ramp measures the fraction of grading criteria each agent satisfies. Finance Jun 2026 62.1% Not in cards
Contextual Invoice OCR Models extract invoice fields using a business’s past bills. Ramp counts invoices where every field matches the submitted bill. Finance Mar 2026 Not in cards
Financial Statement OCR Models extract financial figures from profit and loss statements. Ramp counts values within one percent of a risk operator’s transcription. Finance Mar 2026 Not in cards
Partner Restrictions Compliance Models identify bank partner restrictions that apply to businesses. Ramp scores missed restrictions, coverage across categories, and false positives. Finance Mar 2026 Not in cards