Benchmark Almanac

Miðeind

We track 10 benchmarks that Miðeind made or helped make.

0 in use 10 with public code 10 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
Miðeind Icelandic Case Declension Models decline Icelandic adjective and noun pairs in four cases and two numbers. The public test has 2,400 forms for 300 phrases. Reasoning in many languages Nov 2023 98% Not in cards
Miðeind Icelandic Grammatical Error Detection Models decide whether an Icelandic sentence contains a grammatical error. The test uses 100 correct sentences and 100 incorrect sentences from the Icelandic Error Corpus. Reading in many languages Dec 2023 91.5% Not in cards
Miðeind Icelandic ARC-Challenge Models answer multiple-choice science questions in an Icelandic translation of ARC-Challenge. Miðeind used Claude 3.5 Sonnet to translate the questions. Reasoning in many languages Jun 2024 95.2% Not in cards
Miðeind Icelandic Case Government Models inflect a subject or construct a verb and object phrase in Icelandic. The verb determines the grammatical case. The evaluator checks exact agreement. Reasoning in many languages Sep 2026 100% Not in cards
Miðeind Icelandic Grammar Correction Models correct an Icelandic sentence or return it unchanged. Half the sentences contain one error. The score measures exact agreement with the expected sentence. Chat and instructions in many languages Sep 2026 83.9% Not in cards
Miðeind Icelandic Idioms Models supply a missing word in an Icelandic idiom and explain its meaning. A program checks the missing word. A language model grades the explanation. Reasoning in many languages Sep 2026 85.7% Not in cards
Miðeind Icelandic Popular Culture Models continue Icelandic song lyrics and identify the sources of familiar quotations. The two tasks measure word overlap and exact agreement, respectively. Knowledge in many languages Sep 2026 39.5% Not in cards
Miðeind Icelandic Terminology Models explain Icelandic knitting and crochet terms and name local species from English and Latin names. A model grades definitions. A program checks species names. Knowledge in many languages Sep 2026 78% Not in cards
Icelandic WikiQA Models answer 1,900 factual questions drawn from Icelandic Wikipedia. A language model grades each answer against a human-reviewed reference. Knowledge in many languages Jul 2024 86.6% Not in cards
Miðeind Icelandic Generative Quality Models write paragraphs in Icelandic. Málfríður checks for grammar errors, and a dictionary lookup counts words absent from the Database of Modern Icelandic Inflection. Chat and instructions in many languages Sep 2026 87.7% Not in cards