Benchmark Almanac

Árni Magnússon Institute for Icelandic Studies

We track 5 benchmarks that Árni Magnússon Institute for Icelandic Studies made or helped make.

0 in use 5 with public code 3 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
English to Icelandic Test Suite for Syntactic Divergence Models translate 345 English sentences into Icelandic. Human evaluators judge whether each translation uses an acceptable Icelandic construction for the targeted English syntax. Translation Jun 2026 93.4% Not in cards
SportsEval Models translate 300 English passages about five sports into Icelandic. The scorer checks the Icelandic terms. Humans also judged 37 passages per system at WMT25. Translation Jun 2025 90% Not in cards
WMT25 EEA Terminology Models translate English regulatory sentences into Icelandic. The scorer checks whether they use the official Icelandic terms from the European Economic Area agreement. Translation Jun 2025 46.5% Not in cards
WMT24 Icelandic Idiomatic Expressions Models translate English phrases into Icelandic. Context determines whether a phrase has its everyday literal meaning or its idiomatic meaning. Translation Jun 2024 41.7% Not in cards
WMT24 Icelandic Proper Names Models translate English sentences containing place names or personal names into Icelandic. The scorer checks the Icelandic name and its grammatical form. Translation Jun 2024 Not in cards