University of Illinois Urbana-Champaign
We track 31 benchmarks that University of Illinois Urbana-Champaign made or helped make.
1 in use 27 with public code 24 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| DynaMath Models answer visual math questions generated from 501 seed programs that vary values and graphs. Scored by average and worst-case accuracy across variants. | Visual math and diagrams | Oct 2024 | | Fading |
| SEC-bench Pro Agents hunt and reproduce security bugs in large software projects from limited information. The first release covers V8 and SpiderMonkey. | Finding and exploiting real bugs | May 2026 | | In use |
| MIRAGE-MMST Models answer growers’ questions using photographs and written descriptions. They identify the cause of a problem and recommend how to manage it. | General visual questions | Jun 2025 | 68.0 | Not in cards |
| EmbodiedBench Multimodal LLMs act as embodied agents on 1,128 tasks in four simulators, from household planning (EB-ALFRED, EB-Habitat) to navigation and manipulation. Success rate. | Simulated robots | Feb 2025 | 50.5% | Not in cards |
| DrawingVQA Models answer 92 multiple-choice questions about 33 construction drawings. They read symbols, cross-reference details, and calculate quantities. | Documents, charts and text in images | Apr 2026 | | Not in cards |
| LMBuild Language model agents design 3D structures as parts, joints, materials and assembly steps. Scoring checks that the structures can be built and work as intended. | CAD models | Oct 2026 | 66.5% | Not in cards |
| EmbodiedBench EB-ALFRED Agents follow household instructions through high-level actions in AI2-THOR. The score is the published average task success percentage across EB-ALFRED capability groups. | Simulated robots | Feb 2025 | 67.7% | Not in cards |
| EmbodiedBench EB-Habitat Agents follow instructions to find and rearrange objects through high-level skills in Habitat 2.0. The score is the published average task success percentage. | Simulated robots | Feb 2025 | 58.7% | Not in cards |
| EmbodiedBench EB-Manipulation Agents use visual observations to control a robot arm for object manipulation in simulation. The score is the published average task success percentage. | Simulated robots | Feb 2025 | 28.5% | Not in cards |
| EmbodiedBench EB-Navigation Agents use visual observations and low-level movement actions to reach target objects in AI2-THOR. The score is the published average task success percentage. | Simulated robots | Feb 2025 | | Not in cards |
| AI AgriBench Models answer 416 agronomy questions from agricultural extension publications. Three judges compare each answer with an expert-reviewed reference across four metrics. | Tasks from real jobs | Jan 2026 | 98.1 | Not in cards |
| AdaPlanBench Agents plan under world and user constraints that are revealed step by step during interaction, and must adapt plans as new constraints appear. | General assistants | Jun 2026 | 67.8% | Not in cards |
| PlanBench-XL Tool-use agents solve 327 retail tasks over 1,665 tools that they must first retrieve, chaining calls to uncover evidence. An optional blocking mode simulates tool failures. | Function calling | Jun 2026 | | Not in cards |
| AI AgriBench Smallholders Models answer questions that smallholder farmers sent to advisory services. A judge scores the responses against answers from the advisors. | Tasks from real jobs | 80.9 | Not in cards | |
| CSTutorBench Models tutor students who use block-based programming to control a virtual robot. A rubric grades the feedback on 17 scenarios, including debugging and answer disclosure. | Other professions | Jul 2026 | | Not in cards |
| MonitorBench Models work through 1,540 instances across 19 tasks in 7 categories, each built around a decision-critical factor such as a cue, a hidden goal or a required step. A monitor LLM reads the chain of thought, and the monitorability score measures how often it surfaces that factor. | Deception and alignment | Mar 2026 | 91.9% | Not in cards |
| AI AgriBench Post-Cutoff Models answer 146 questions from newer agricultural extension documents. Judges score their answers for accuracy, relevance, completeness and conciseness. | Tasks from real jobs | Jan 2026 | 98.1 | Not in cards |
| OpenFrontiers-Bench Agents try to rediscover proofs of research math problems, most resolved during 2026, stated in Lean 4. Two independent checkers grade the formal proofs, with closed-book, frozen-retrieval and live-web settings. | Formal proofs | Oct 2026 | 42.2% | Not in cards |
| ReactionBench Models rank a viewer’s reactions, match a reaction to one of four video clips, and explain which cue prompted it. | Video understanding | Oct 2026 | 0.51 | Not in cards |
| VGI-Bench Video generation models solve 27 visual reasoning tasks with 810 instances through generated frames. Scoring checks that the process is valid, not only the final frame. | Video world models | Aug 2026 | | Not in cards |
| MIRAGE-MMMT Models decide whether to ask for more information or answer a grower during an agricultural consultation. They then write the question or answer. | Multi-turn conversation | Jun 2025 | 65.5% | Not in cards |
| SEC-bench Pro 260617 Agents hunt security bugs in large software projects. This snapshot adds Linux kernel instances and more SpiderMonkey instances to the first release. | Finding and exploiting real bugs | Jun 2026 | 58.4% | Not in cards |
| ProtocolLLM Models write SystemVerilog modules for four communication protocols. The authors check syntax, synthesis, and whether simulated signals meet protocol timing requirements. | Electronics | Jun 2025 | | Not in cards |
| RefuNQ Models answer factual questions and decide when to decline. The authors create unanswerable questions by replacing a noun in a real question with an invented term. | Short factual questions | Nov 2023 | 53.7% | Not in cards |
| Fire360 Models interpret panoramic video from firefighter training drills. They answer questions, caption actions, assess procedures, locate equipment, and match damaged objects to clean examples. | Video understanding | Jun 2025 | 53.8% | Not in cards |
| SEC-bench Code agents work on real OSS-Fuzz vulnerabilities in containerized repositories. They either write a proof-of-concept input that reproduces the crash or patch the code so the crash stops. | Finding and exploiting real bugs | Jun 2025 | 18% | Not in cards |
| GTFS Retrieval Models query a Chicago bus schedule from four routes. They answer directly or generate Python code whose output is compared with a reference answer. | Expert-written questions | Jul 2023 | Not in cards | |
| GTFS Semantics Models answer multiple-choice questions about the General Transit Feed Specification. Questions cover transit data files, field definitions, and relationships between tables. | Expert-written questions | Jul 2023 | Not in cards | |
| Minecraft Builder Action Prediction Models use dialogue and the current block layout to predict which blocks a Minecraft builder will place or remove. Net action F1 measures agreement with human moves. | Embodied reasoning | Jul 2020 | Not in cards | |
| Mobile-Eval-E Agents complete 25 smartphone tasks across 15 apps. Human reviewers score the share of task requirements each agent meets. | Phones | Jan 2025 | Not in cards | |
| TailOR Models generate images and videos of regular, unconventional, and impossible tool interactions. Annotators score instruction adherence, interaction accuracy, physical realism, and perceptual quality. | Physical prediction | Jun 2026 | Not in cards |