Writer
We track 5 benchmarks that Writer made or helped make. Its models are on its lab page.
0 in use 0 with public code 1 with a leaderboard here
Benchmarks
| Name | Kind | Released | Best score | Status |
|---|---|---|---|---|
| FailSafeQA Models answer finance questions over long documents while the query or the document is degraded, irrelevant or missing. An LLM judge scores robustness, context grounding and compliance. | Finance | Feb 2025 | | Not in cards |
| Current-facts set Models answer questions about current facts. Writer evaluates responses as part of its neutrality and safety study. | Short factual questions | Aug 2026 | Not in cards | |
| OmniACT Models generate executable scripts from a screenshot and a user request. The evaluation compares the actions needed to complete desktop and web tasks. | Desktop apps | Feb 2024 | Not in cards | |
| Paired-symmetry suite Models respond to paired political prompts. Writer compares refusal behavior across the paired requests. | Bias, fairness and politics | Aug 2026 | Not in cards | |
| Palmyra X6 Enterprise Workflow Evaluations Agents perform production workflows that require retrieval, tool use, content generation, or delegation. Writer compares capability and cost. | Running a business | Aug 2026 | Not in cards |