Benchmark Almanac

University of Gothenburg

We track 12 benchmarks that University of Gothenburg made or helped make.

0 in use 9 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
DaLAJ-GED 2.0 Models decide whether a Swedish sentence contains a grammatical error. The test includes learner writing and examples from Swedish coursebooks. Reading in many languages Feb 2023 Not in cards
SuperSim Models give separate scores for how similar two Swedish words are and how closely they are associated. Five Swedish speakers rated each pair. Reading in many languages Apr 2021 Not in cards
SweDiagnostics Models judge entailment between Swedish sentences that test phenomena such as negation, quantifiers, and ambiguity. Reasoning in many languages Jun 2021 Not in cards
SweFAQ 2.0 Models match questions from Swedish public agencies to answers from the same subject category. Reading in many languages Sep 2022 Not in cards
SweNLI Models decide whether Swedish premises entail a statement, contradict it, or leave it undecided. Reasoning in many languages Feb 2023 Not in cards
SweParaphrase 1.0 Models rate the similarity of two Swedish sentences. Correlation measures agreement with scores from the English source pairs. Reading in many languages Jun 2021 Not in cards
SweParaphrase 2.0 Models rate meaning overlap between Swedish sentences on a scale from zero to five. Reading in many languages Oct 2022 Not in cards
SweSAT Synonyms 1.0 Models choose the correct synonym or definition from five options on the Swedish university entrance exam. Reading in many languages Jun 2021 Not in cards
SweSAT Synonyms 1.1 Models choose one of five Swedish synonyms or definitions. The official score adjusts accuracy for random guessing. Reading in many languages Mar 2023 Not in cards
SweWiC Models decide whether a highlighted Swedish word has the same meaning in two sentences. Reading in many languages Jun 2021 Not in cards
SweWinogender 2.0 Models judge whether a Swedish pronoun refers to a person in a stated occupation. The parity score checks agreement across three pronoun variants. Bias, fairness and politics Feb 2023 Not in cards
SweWinograd 2.0 Models decide whether a highlighted Swedish pronoun refers to a proposed person or thing in the passage. Reading in many languages Feb 2023 Not in cards