Benchmark Almanac

Språkbanken Text

We track 8 benchmarks that Språkbanken Text made or helped make.

0 in use 6 with public code 0 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
SuperLim 1.0 Models answer Swedish language questions about word meaning, sentence similarity, inference, and references to people or things. Reading in many languages May 2021 Not in cards
SuperLim 2.0 Models complete 15 Swedish language tasks across 14 datasets. Tasks include recognizing arguments, choosing synonyms, and summarizing newspaper articles. Reading in many languages Sep 2022 Not in cards
SweFAQ 2.0 Models match questions from Swedish public agencies to answers from the same subject category. Reading in many languages Sep 2022 Not in cards
SweNLI Models decide whether Swedish premises entail a statement, contradict it, or leave it undecided. Reasoning in many languages Feb 2023 Not in cards
SweSAT Synonyms 1.0 Models choose the correct synonym or definition from five options on the Swedish university entrance exam. Reading in many languages Jun 2021 Not in cards
SweSAT Synonyms 1.1 Models choose one of five Swedish synonyms or definitions. The official score adjusts accuracy for random guessing. Reading in many languages Mar 2023 Not in cards
SweWinogender 2.0 Models judge whether a Swedish pronoun refers to a person in a stated occupation. The parity score checks agreement across three pronoun variants. Bias, fairness and politics Feb 2023 Not in cards
SweWinograd 2.0 Models decide whether a highlighted Swedish pronoun refers to a proposed person or thing in the passage. Reading in many languages Feb 2023 Not in cards