CTI-to-MITRE
Models classify a threat description using MITRE ATT&CK technique labels. The HELM scenario presents a set of candidate labels.
- Made by
- Vittorio Orbinato and colleagues
- Released
- August 2022
- Kind
- Secure code and review
- Level
- University
- Scored
- Checked automatically
- Answers
- Multiple choice
- Tasks
- All public
- Measures
- Capability
- Language
- English
- Interaction
- Single turn
- Refresh
- One fixed set
- Run by
- Anyone
SecureBERT tops the leaderboard with 72.5%
Best score per entry from the benchmark's paper, read on November 20th, 2026. A paper’s table doesn’t add newer models. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.
- 1 SecureBERT 72.5%
- 2 MLP 70.4%
- 3 SVM (OvO) 69.9%
- 4 SVM (OvR) 69.9%
- 5 Logistic Regression (balanced) 64.6%
- 6 Complement Naive Bayes 63.9%
- 7 CNN 61.4%
- 8 LSTM (Word2vec) 61%
- 9 LSTM 57.7%
- 10 Logistic Regression 55.6%
- 11 Multinomial Naive Bayes 36.8%
Source: The CTI-to-MITRE paper
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | SecureBERT | 72.5% | Official leaderboard | ||
| 2 | MLP | 70.4% | Official leaderboard | ||
| 3 | SVM (OvO) | 69.9% | Official leaderboard | ||
| 4 | SVM (OvR) | 69.9% | Official leaderboard | ||
| 5 | Logistic Regression (balanced) | 64.6% | Official leaderboard | ||
| 6 | Complement Naive Bayes | 63.9% | Official leaderboard | ||
| 7 | CNN | 61.4% | Official leaderboard | ||
| 8 | LSTM (Word2vec) | 61% | Official leaderboard | ||
| 9 | LSTM | 57.7% | Official leaderboard | ||
| 10 | Logistic Regression | 55.6% | Official leaderboard | ||
| 11 | Multinomial Naive Bayes | 36.8% | Official leaderboard |
No lab model card reports it yet
We read the headline results of 540 launch posts from 57 labs, and none of them list this benchmark.
More in Cybersecurity
- ExploitGym In use 869 real-vulnerability exploit-development tasks (userspace, V8, Linux kernel). Success means capturing a flag through the intended vulnerability. GPT-6 Astra 42.4%
- ExploitBench In use 41 recent V8 JavaScript-engine N-day vulnerabilities. Progress measured by 16 grader-verified capability flags across five tiers. GPT-6 Astra 100%
- ExploitBench - Internal Port In use Agents develop working exploits for recently disclosed vulnerabilities to limit contamination. Scored as arbitrary code execution success rate. GPT-6.2 Astra 53.5%
- Atomic Challenges In use Agents solve 22 skill-level offensive security challenges in vulnerability research, network attack simulation and evasion. Scored as challenges solved. GPT-6 Astra 90.9%
- MiMo Cyber Bench In use Agents do in-house cybersecurity tasks that add to public vulnerability reproduction and exploitation benchmarks. The report gives no task count or grading details. MiMo-V2.6 Pro 81.7%
- PatchBench (OpenAI) In use Models patch security flaws. Automated tests and model-based review check that each patch fixes the flaw, keeps the intended behavior and is ready to merge. GPT-6.2 Astra 79.7%
Data audit
All 8 checks pass. Every benchmark
Catalog record 2 pass
- Pass Task access All tasks are public.
- Pass Confidence Checked against a primary source.
Scores 2 pass 3 notes
- Pass Independent runs Scores from the benchmark’s paper (11 entries).
- Pass Models scored 11 entries on its leaderboard, none a model we track.
- Note Newest score No scores yet.
- Note Frontier models No models scored yet.
- Note Headroom No scores yet.
Leaderboard import 3 pass
- Pass Importer We read the board on arxiv.org: 11 entries at the last read, none a model we track.
- Pass Last read November 20th, 2026.
- Pass Read worked The last read worked.
Consistency 1 pass 1 note
- Note Lab claims No model has both a lab number and an independent run.
- Pass Known issues No published audit found problems.