Benchmark Almanac

HKUST

We track 7 benchmarks that HKUST made or helped make.

0 in use 7 with public code 5 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
CodeEditorBench Models edit code in four scenarios: debugging, translation, requirement switching and polishing. Unit tests check the edits, and polishing also measures speed and memory gains. Function-level problems Apr 2024 85.5% System cards only
Accelerate Bench Coding agents speed up 32 real scientific applications, such as GROMACS and OpenFOAM, on a fixed CPU allocation within 8 hours. A hidden verifier checks correctness, and Score@1 averages the share of runtime saved, with incorrect programs scoring zero. Speed and optimization Nov 2026 67.4% Not in cards
AdaExploreBench Language model agents explore environments with hidden rules, infer the rules from evidence and adapt when the rules change without notice. Puzzles and chess Oct 2026 70.3% Not in cards
VWE-Bench Multimodal agents build interactive 3D worlds from a user request. They infer intent, plan the scene layout, call 3D tools for asset retrieval, editing and rendering, and revise from rendered feedback. The benchmark has 2,616 3D assets, 323 human-annotated seed worlds and 6,828 queries. Verified queries have ground truth, and the rest are scored with rubrics. CAD models Aug 2026 57.3% Not in cards
FollowBench Models answer 820 English and Chinese instructions where each level adds one more constraint of content, situation, style, format or example type. Rules and an LLM judge check each constraint. Following precise instructions Oct 2023 73.4% Not in cards
FortisAVQA Models answer questions about sounds and visible events in music videos. Accuracy is reported separately for frequent and rare question distributions after the original questions are rephrased. Video understanding Apr 2025 48.7% Not in cards
JailTrickBench Models respond to jailbreak attacks under varied attack and defense settings. A GPT-4 evaluator judges attack success across harmful-request datasets. Jailbreaks and prompt injection Jun 2024 Not in cards