Qatar Computing Research Institute
We track 10 benchmarks that Qatar Computing Research Institute made or helped make.
0 in use 7 with public code 1 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| AraDiCE Models are tested on Arabic dialect understanding and generation and on cultural awareness of the Gulf, Egypt and the Levant. Datasets were machine translated into dialects and post-edited by people. | Knowledge in many languages | Sep 2024 | System cards only | |
| IslamicFaithQA Models answer short Islamic knowledge questions in Arabic and English. A model judges their answers against reference answers. | Expert-written questions | Jan 2026 | 21.6% | Not in cards |
| xCodeEval Models understand, generate, translate, and retrieve contest code across seven tasks and up to 17 programming languages. Generated programs run against test cases through ExecEval. | Competitive programming | Mar 2023 | | Not in cards |
| AraBench Models translate dialectal Arabic into English. The suite covers travel conversations, media transcripts and religious texts. Researchers score translations against references with BLEU. | Translation | Dec 2020 | Not in cards | |
| IslamicEval 2025 Quotation Correction Models correct inaccurate Quran and Hadith quotations against the supplied sources. The organizers score exact corrections and accept a designated answer for uncorrectable text. | Faithful to sources | Jun 2025 | Not in cards | |
| IslamicEval 2025 Quotation Span Identification Models mark the character spans of intended Quran and Hadith quotations in Arabic model answers. Organizers score character labels with macro F1. | Faithful to sources | Jun 2025 | Not in cards | |
| IslamicEval 2025 Quotation Validation Models check whether marked Quran and Hadith quotations agree with their sources. The organizers report accuracy. | Faithful to sources | Jun 2025 | Not in cards | |
| IslamicEval 2025 Quran and Hadith Passage Retrieval Models rank up to 20 passages from the Quran and Sahih al-Bukhari for an Arabic question. Organizers score mean average precision at ten. | Questions that need search | Jun 2025 | Not in cards | |
| PalmX 2025 General Culture Models choose answers about customs, history and daily life across Arab countries. The test contains 2,000 questions in Modern Standard Arabic. | Knowledge in many languages | Jun 2025 | Not in cards | |
| PalmX 2025 Islamic Culture Models choose answers about Islamic practices, scripture and history in Modern Standard Arabic. The organizers score exact answer choices. | Knowledge in many languages | Jun 2025 | Not in cards |