Benchmark Almanac

Andon Labs

We track 6 benchmarks and 2 arenas that Andon Labs made or helped make.

2 in use 0 with public code 8 with a leaderboard here

Arena leaderboards

Top five in each of its 2 arenas with three or more rated models, the most rated first.

Benchmarks

Name Kind Released Best score Status
Vending-Bench 2 Agent runs a simulated vending-machine business for one year starting with $500. Score is the final bank balance, averaged over runs. Running a business Nov 2025 $16,843 In use
Blueprint-Bench 2 Agents turn about 20 interior photos of each of 50 apartments into 2D floor plans. Scores compare room connectivity with ground truth, normalized against random. Spatial reasoning and counting May 2026 58.6% In use
Vending-Bench Agents run a simulated vending machine business over long horizons, ordering stock, setting prices, and paying daily fees. Scoring is final net worth in dollars. Running a business Feb 2025 $4,694 Dropped
Drone-Bench Agent writes code for a small real drone across five tasks (reconstruct, localize, navigate, detect, follow). Each scored against a human-written baseline. Real robots Jul 2026 97.3% Not in cards
Blueprint-Bench Models turn about 20 interior photos of each of 50 apartments into a 2D floor plan. Scored by similarity of room connectivity and size ranking to the true plan. Spatial reasoning and counting Sep 2025 0.43 Not in cards
Butter-Bench A language model controls a real robot to find and deliver a pack of butter in an office. It is scored on completing each step of the task. Real robots Oct 2025 37% Not in cards