Benchmark Almanac

National University of Singapore

We track 18 benchmarks that National University of Singapore made or helped make.

1 in use 17 with public code 8 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
MixEval Models answer real user queries matched to similar questions from existing benchmarks. Ground-truth answers make it a cheap proxy for Chatbot Arena rankings. Overall chat quality Jun 2024 89.9% Dropped
NExT-QA Models answer causal, temporal and descriptive questions about short videos of daily life. Labs usually report the multiple choice split. Video understanding May 2021 86.2% Dropped
Last Secure Code Benchmark Agents rebuild code that a real vulnerability once affected, from a specification that never mentions security. Hidden tests check function and replay an exploit. Secure code and review Oct 2026 25.1% In use
TAT-DQA Models answer questions about financial report pages that mix tables and text, often needing arithmetic. Scored by exact match and F1. Documents, charts and text in images Jul 2022 37.8% Dropped
General-Bench Models perform multimodal understanding and generation tasks. General-Level averages task scores and measures performance above specialist baselines at progressively broader levels. General visual questions May 2025 8.41 Not in cards
ViMU Models explain what a short video implies and select the cues that support their reading. They also identify rhetorical devices and social signals. Video understanding May 2026 46.9% Not in cards
Learn2Play Bench Agents play unfamiliar games whose rules are not given and must learn from experience across episodes. Improvement over repeated play is the measure. Puzzles and chess Oct 2026 81.8 Not in cards
EgoBlind Models answer questions from blind users using only video recorded before each question. GPT-4o mini checks each answer against human references. Video understanding Mar 2025 59.3% Not in cards
CutVerse Agents complete editing tasks in seven applications, including Premiere Pro, Photoshop, and DaVinci Resolve. Vision models grade progress against milestones in the recorded workflow. Desktop apps May 2026 68.3% Not in cards
RSIBench-Data Coding agents act as data-centric researchers. With a fixed Tinker post-training stack and budget, they refine training data from dev-benchmark feedback, and the final model is scored on held-out test benchmarks. Machine learning engineering Jul 2026 32.6% Not in cards
SO-Arena Models choose answers across 11 domains and five capability groups. Evaluation tests accuracy, stability under changed option sets, and recognition that the correct answer is missing. Language reasoning Nov 2026 82.3% Not in cards
WorldGUI Desktop GUI agents do 611 tasks that start from varied initial states, such as the target app not being open. Tasks come with instructional videos and project files. Desktop apps Feb 2025 45.8% Not in cards
EndoVis18-VQA Models answer questions about organs, tools, tool positions, and interactions in images from robotic kidney surgery. The benchmark accepts class labels or sentence answers in separate tasks. Medicine Jun 2022 Not in cards
MultiRef-Compass Models generate audio-video content from several visual references and text instructions across 350 samples. Fourteen metrics assess quality, reference fidelity, audio-visual consistency, and instruction following. The evaluator combines automatic measurements with multimodal model judgments. Video generation Jul 2026 Not in cards
MyCareersFuture Skill Retrieval Models rank required skills from job descriptions on Singapore’s government employment portal. Employer tags supply the answers, including skills the description leaves implicit. Other professions Dec 2020 Not in cards
PiRhDy Accompaniment Suggestion Models rank 50 accompaniment phrases for a MIDI melody. Several tracks can belong with the melody. The authors report mean average precision. Audio understanding and voice Oct 2020 Not in cards
PiRhDy Melody Completion Models rank 50 possible continuations of a MIDI melody. One phrase follows the prompt in the original piece. The authors report mean average precision. Audio understanding and voice Oct 2020 Not in cards
TAT-QA Models answer 16,552 questions about 2,757 passages from real financial reports that mix a table with text. Many answers need arithmetic, counting or comparison. Scored by exact match and F1. Finance May 2021 Not in cards