Benchmark Almanac

DeepMind

We track 3 benchmarks that DeepMind made or helped make.

0 in use 2 with public code 1 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
WSC Models decide whether a pronoun refers to a given noun phrase in sentences that need commonsense to resolve. Scored by accuracy. Commonsense May 2019 86.9% Dropped
SuperGLUE Models solve eight English language understanding tasks, including BoolQ, COPA, MultiRC, ReCoRD, RTE, WiC, and WSC. Scored by the average task score. Language reasoning May 2019 90.4% Not in cards
GLUE Models predict labels or similarity scores across nine English language-understanding tasks. The overall score averages task scores. Tasks with several metrics first average those metrics. Language reasoning Apr 2018 Not in cards