Benchmark Almanac

CLEF JOKER organizers

We track 13 benchmarks that CLEF JOKER organizers made or helped make.

0 in use 13 with public code 2 with a leaderboard here

Benchmarks

Name Kind Released Best score Status
CLEF JOKER 2026 Task 4: Humour Generation Models write a joke using a target word in two specified senses. Three language models compare pairs of jokes for each language ranking. Creative writing Oct 2025 1,389 Not in cards
CLEF JOKER 2026 Task 3: Onomastic Wordplay Translation Models translate English names containing wordplay into French using a short description. The official score accepts reference matches and alternatives approved by translators. Translation Oct 2025 71.9% Not in cards
CLEF JOKER 2023 Task 1: Pun Detection Models decide whether a text contains a pun. Organizers report precision, recall, F1, and accuracy separately for English, French, and Spanish. Reading in many languages Sep 2022 55% Not in cards
CLEF JOKER 2023 Task 3: Pun Translation Models translate English puns into French or Spanish. Judges count translations that retain both the original meaning and wordplay. Organizers rank each language separately. Translation Sep 2022 10.0 Not in cards
CLEF JOKER 2023 Task 2: Pun Location and Interpretation Models identify the punning word in English, French, or Spanish jokes. The English interpretation test also asks for words that express both meanings. Reasoning in many languages Sep 2022 Not in cards
CLEF JOKER 2024 Task 1: Humour-Aware Information Retrieval Systems rank short English texts for each query. A relevant text must match the topic and contain humor. The organizers report mean average precision. Questions that need search Sep 2023 Not in cards
CLEF JOKER 2024 Task 2: Humour Classification According to Genre and Technique Models assign one of six humor techniques to an English text. Organizers rank runs by accuracy and report F1 for each technique. Reading comprehension Sep 2023 Not in cards
CLEF JOKER 2024 Task 3: Pun Translation Models translate English puns into French. Organizers compare their output with 832 human translations of 376 source texts using BLEU and BERTScore. Translation Nov 2023 Not in cards
CLEF JOKER 2025 Task 1: Humour-Aware Information Retrieval Systems retrieve texts that match a query and contain humor. Organizers report mean average precision separately for English and European Portuguese. Questions that need search Oct 2024 Not in cards
CLEF JOKER 2025 Task 2: Wordplay Translation Models translate English puns into French. A translator judges whether each sampled answer preserves the meaning and contains wordplay. The online leaderboard uses BLEU. Translation Oct 2024 Not in cards
CLEF JOKER 2025 Task 3: Onomastic Wordplay Translation Models translate English names containing wordplay into French using a description. The organizers check exact matches and have translators assess alternatives. Translation Oct 2024 Not in cards
CLEF JOKER 2026 Task 1: Humour-Aware Information Retrieval Systems retrieve humorous passages that match a topic in English or Hinglish. Organizers rank systems by mean average precision. Questions that need search Oct 2025 Not in cards
CLEF JOKER 2026 Task 2: Pun Translation Models translate English puns into French. The automatic score checks whether a translation contains an annotated French word or phrase that carries the pun. Translation Oct 2025 Not in cards