Benchmark Almanac

Norman Mu

We track 3 benchmarks that Norman Mu made or helped make.

0 in use 3 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
RuLES Models follow explicit rules while talking with a user. Each scenario checks automatically whether the model broke a rule. Following precise instructions Nov 2023 9.54% Not in cards
RuLES 2.0 Models follow explicit rules while talking with a user. Each scenario checks automatically whether the model broke a rule. Following precise instructions Mar 2024 9.54% Not in cards
SLIP zero-shot transfer suite Models assign labels to images using text descriptions of the classes. The evaluation averages zero-shot accuracy across 26 recognition tasks. General visual questions Dec 2021 Not in cards