Benchmark Almanac

Max Planck Institute for Intelligent Systems

We track 2 benchmarks that Max Planck Institute for Intelligent Systems made or helped make.

0 in use 1 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
SGP-Bench Models read symbolic graphics programs such as SVG and CAD code and answer questions about what the rendered image would show, without seeing the image. General visual questions Aug 2024 67.4% Dropped
QuantSightBench Models give 90% prediction intervals for 1,000 numerical forecasting questions with known outcomes. The main setting allows retrieval from a fixed news corpus. Mean Log Interval Score penalizes wide intervals and missed outcomes, while coverage measures how often intervals contain the answer. Real-world events Apr 2026 6.70 Not in cards