Benchmark Almanac

RelayForge

We track 5 benchmarks that RelayForge made or helped make.

0 in use 0 with public code 4 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Instrumentation Universal Knowledge v1.6 refresh Models answer 60 technician questions and 97 specialist questions from the June samples. Three model judges score responses under higher output limits. Tasks from real jobs Oct 2026 93.3% Not in cards
Instrumentation Universal Knowledge v1.3 Models answer 60 industrial instrumentation questions. Five models qualified for a longer foundation test and two advanced tests. Model judges grade each answer. Tasks from real jobs Jun 2026 93.3% Not in cards
Instrumentation Universal Knowledge v1.5 Models diagnose instrumentation faults and answer industrial controls questions across four tiers. The composite weights each tier by its difficulty and question count. Tasks from real jobs Jun 2026 86.4% Not in cards
MANDOS Session 2 Models make process safety decisions across four private industrial scenarios. A model judge assesses each run. The leaderboard reports the share of passing runs. Tasks from real jobs Sep 2026 96.9% Not in cards
MANDOS Run 003 Models make decisions during seven industrial incidents and four fictional scenarios. A model judge checks their decisions, including whether they seek expert advice. Tasks from real jobs Jun 2026 100% Not in cards