referee-lab
We track 3 benchmarks that referee-lab made or helped make.
0 in use 0 with public code 0 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| Effective FPS Models decide whether video frames are consecutive or duplicated to simulate a higher capture rate. The evaluator checks each prediction against the construction of the clip. | Video understanding | Aug 2026 | Not in cards | |
| Escalation Router Models predict whether Qwen2.5-VL 7B correctly identifies visible hands or grippers in a clip. The reference label comes from two observed runs per clip. | Video understanding | Aug 2026 | Not in cards | |
| Hands Visible Models inspect 12 frames per clip and determine whether a human hand or robot gripper appears. The 45-clip evaluation uses provisional machine-generated labels. | Video understanding | Aug 2026 | Not in cards |