Benchmark Almanac

Tacit Knowledge and Troubleshooting

GPT-6 Luna 94% leads since October 2026
Only in system cards

Models answer multiple-choice questions on tacit lab knowledge and troubleshooting that need hands-on expertise. Scored as cons@32.

OpenAI Preparedness evaluation. The questions are not public.

Released
August 2024
Kind
Lab protocols
Level
Expert
Scored
Checked automatically
Answers
Multiple choice
Tasks
Kept private
Strongest model tested
GPT-6 Astra #3 on the capability index
Last in a system card
October 2026
Last model added
October 2026
30
Models scored in lab cards
39
System cards report it, from 1 lab

GPT-6 Luna leads with 94%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

30 models. Scroll sideways for the rest.

  1. 94.0 GPT-6 Luna
  2. 88.5 GPT-6.1 Sol
  3. 84.1 GPT-5.6 Terra
  4. 83.0 GPT-5.6 Luna
  5. 82.2 GPT-5.6 Sol
  6. 81.7 GPT-5.5 Pro
  7. 79.2 GPT-6 Sol
  8. 78.3 GPT-5.5
  9. 78.3 GPT-Rosalind-6
  10. 77.0 GPT-5.1-Codex-Max
  11. 74.0 gpt-oss-120b
  12. 73.0 gpt-oss-20b
  13. 73.0 GPT-5.2 Codex
  14. 72.0 OpenAI deep research
  15. 72.0 GPT-4.5
  16. 72.0 GPT-5.3 Codex
  17. 71.7 GPT-5.4 Mini
  18. 70.0 o3
  19. 69.0 GPT-4o
  20. 68.9 GPT-5.5 Instant
  21. 68.0 o4-mini
  22. 68.0 GPT-5 mini
  23. 66.7 o1-mini
  24. 66.7 o1-preview
  25. 66.0 o1
  26. 66.0 GPT-5
  27. 65.0 GPT-5.4
  28. 63.3 GPT-6 Astra
  29. 58.0 o3-mini
  30. 48.0 GPT-5.2
  1. 1 GPT-6 Luna 94%
  2. 2 GPT-6.1 Sol 88.5%
  3. 3 GPT-5.6 Terra 84.1%
  4. 4 GPT-5.6 Luna 83%
  5. 5 GPT-5.6 Sol 82.2%
  6. 6 GPT-5.5 Pro 81.7%
  7. 7 GPT-6 Sol 79.2%
  8. 8 GPT-5.5 78.3%
  9. 9 GPT-Rosalind-6 78.3%
  10. 10 GPT-5.1-Codex-Max 77%
  11. 11 gpt-oss-120b 74%
  12. 12 gpt-oss-20b 73%
  13. 13 GPT-5.2 Codex 73%
  14. 14 OpenAI deep research 72%
  15. 15 GPT-4.5 72%
  16. 16 GPT-5.3 Codex 72%
  17. 17 GPT-5.4 Mini 71.7%
  18. 18 o3 70%
  19. 19 GPT-4o May 2024 69%
  20. 20 GPT-5.5 Instant 68.9%
  21. 21 o4-mini 68%
  22. 22 GPT-5 mini 68%
  23. 23 o1-mini 66.7%
  24. 24 o1-preview 66.7%
  25. 25 o1 66%
  26. 26 GPT-5 66%
  27. 27 GPT-5.4 65%
  28. 28 GPT-6 Astra 63.3%
  29. 29 o3-mini 58%
  30. 30 GPT-5.2 48%

Source: Labs’ model cards

#ModelScoreSettingDateSource
1 GPT-6 Luna 94% Oct 2026 Lab model card
2 GPT-6.1 Sol 88.5% Sep 2026 Lab model card
3 GPT-5.6 Terra 84.1% Jun 2026 Lab model card
4 GPT-5.6 Luna 83% Aug 2026 Lab model card
5 GPT-5.6 Sol 82.2% Aug 2026 Lab model card
6 GPT-5.5 Pro 81.7% Apr 2026 Lab model card
7 GPT-6 Sol 79.2% Sep 2026 Lab model card
8 GPT-5.5 78.3% Apr 2026 Lab model card
9 GPT-Rosalind-6 78.3% Oct 2026 Lab model card
10 GPT-5.1-Codex-Max 77% Nov 2025 Lab model card
11 gpt-oss-120b 74% Aug 2025 Lab model card
12 gpt-oss-20b 73% Aug 2025 Lab model card
13 GPT-5.2 Codex 73% Dec 2025 Lab model card
14 OpenAI deep research 72% Feb 2025 Lab model card
15 GPT-4.5 72% Feb 2025 Lab model card
16 GPT-5.3 Codex 72% Feb 2026 Lab model card
17 GPT-5.4 Mini 71.7% Mar 2026 Lab model card
18 o3 70% Apr 2025 Lab model card
19 GPT-4o May 2024 69% Aug 2024 Lab model card
20 GPT-5.5 Instant 68.9% May 2026 Lab model card
21 o4-mini 68% Apr 2025 Lab model card
22 GPT-5 mini 68% Aug 2025 Lab model card
23 o1-mini 66.7% Sep 2024 Lab model card
24 o1-preview 66.7% Sep 2024 Lab model card
25 o1 66% Dec 2024 Lab model card
26 GPT-5 66% Aug 2025 Lab model card
27 GPT-5.4 65% Mar 2026 Lab model card
28 GPT-6 Astra 63.3% Sep 2026 Lab model card
29 o3-mini 58% Jan 2025 Lab model card
30 GPT-5.2 48% Dec 2025 Lab model card

8 models have held number one in labs’ own numbers since its release in August 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Lab claims Passed 85% in Sep 2026, by GPT-6.1 Sol
GPT-6 Luna 94%

Source: Labs’ model cards

OpenAI’s own number for GPT-6.1 Sol passed 85% in September 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) GPT-6 Luna GPT-6.1 Sol GPT-5.6 Terra GPT-5.1-Codex-Max OpenAI deep research GPT-4o 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) GPT-6 Luna GPT-5.6 Terra

Source: Labs’ model cards

Labs report it only in system cards

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark. Some system cards do, below.

39 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
Oct 30, 2026 GPT-Rosalind-6 system card 78.3 Tacit Knowledge and Troubleshooting, cons@32 (refusal-adjusted 85.0%), Table 4 Biorisk knowledge Bio and chem risk
Oct 29, 2026 GPT-6 Luna system card addendum 90.8 Tacit Knowledge and Troubleshooting, cons@32 (Table 8) Bio and chem risk
Oct 7, 2026 GPT-6 Sol system card addendum 72.5 Tacit Knowledge and Troubleshooting, cons@32 (Table 8) Bio and chem risk
Oct 7, 2026 GPT-6 Luna system card addendum 94 Tacit Knowledge and Troubleshooting, cons@32 (Table 8) Bio and chem risk
Sep 29, 2026 GPT-6.1 Sol system card addendum 88.5 Tacit Knowledge and Troubleshooting, refusal-adjusted cons@32 (9.1.1.1) Bio and chem risk
Sep 22, 2026 GPT-6 Sol system card 79.2 Tacit Knowledge and Troubleshooting, cons@32 (Table 33, A.8.1.1.1) Bio and chem risk
Sep 22, 2026 GPT-6 Luna system card 69.2 Tacit Knowledge and Troubleshooting, cons@32 (Table 33, A.8.1.1.1) Bio and chem risk
Sep 3, 2026 GPT-6 Astra system card 63.3 Tacit Knowledge and Troubleshooting, cons@32 (refusal-adjusted 90.00%) (10.1.1.1.3) Bio and chem risk
Aug 6, 2026 GPT-5.6 Sol system card addendum 82.2 Tacit Knowledge and Troubleshooting, cons@32 with refusals or safe completions counted (78.81% without), printed chart label (Figure 5, 7.1.1.3) Bio and chem risk
Aug 6, 2026 GPT-5.6 Luna system card addendum 83 Tacit Knowledge and Troubleshooting, cons@32 with refusals or safe completions counted (76.67% without), printed chart label (Figure 5, 7.1.1.3) Bio and chem risk
Jul 9, 2026 GPT-5.6 Terra system card 84.1 Tacit Knowledge and Troubleshooting, refusals and safe completions counted as successes, stated in text (9.1.1.3) Bio and chem risk
Jul 9, 2026 GPT-5.6 Sol system card Tacit Knowledge and Troubleshooting, chart without printed numbers (Figures 26, 27, 9.1.1.3) Bio and chem risk
Jul 9, 2026 GPT-5.6 Luna system card Tacit Knowledge and Troubleshooting, chart without printed numbers (Figures 26, 27, 9.1.1.3) Bio and chem risk
Jun 25, 2026 GPT-5.6 Terra system card 84.1 Tacit Knowledge and Troubleshooting, refusals and safe completions counted as successes, stated in text (9.1.1.3) Bio and chem risk
Jun 25, 2026 GPT-5.6 Sol system card Tacit Knowledge and Troubleshooting, chart without printed numbers (Figures 26, 27, 9.1.1.3) Bio and chem risk
Jun 25, 2026 GPT-5.6 Luna system card Tacit Knowledge and Troubleshooting, chart without printed numbers (Figures 26, 27, 9.1.1.3) Bio and chem risk
May 4, 2026 GPT-5.5 Instant system card 68.9 Tacit Knowledge and Troubleshooting, cons@32 (80.79% counting refusals or safe completions), printed chart label (Figure 5, 8.1.1.3) Bio and chem risk
Apr 23, 2026 GPT-5.5 system card 78.3 Tacit Knowledge and Troubleshooting, cons@32, printed chart label (Figure 14, 9.1.1.3) Bio and chem risk
Apr 23, 2026 GPT-5.5 Pro system card 81.7 Tacit Knowledge and Troubleshooting, cons@32 excluding refusals, GPT-5.5 Pro, text only, not pictured (9.1.1.3) Bio and chem risk
Mar 17, 2026 GPT-5.4 Mini system card 71.7 Tacit Knowledge and Troubleshooting, cons@32 (77.77% counting refusals) (Table 18) Bio and chem risk
Mar 5, 2026 GPT-5.4 system card 65 Tacit Knowledge and Troubleshooting, cons@32 excluding refusals (83.8% counting refusals and safe completions) (Figure 10, 5.1.1.3) Bio and chem risk
Feb 5, 2026 GPT-5.3 Codex system card 72 Tacit knowledge and troubleshooting, cons@32, printed chart label, refusals counted as failures (Figure 1, 5.1.1.1) Bio and chem risk
Dec 18, 2025 GPT-5.2 Codex system card addendum 73 Biorisk Tacit Knowledge and Troubleshooting, cons@32, printed chart label (Figure 3, 5.1.1.3) Bio and chem risk
Dec 11, 2025 GPT-5.2 system card addendum 48 Biorisk Tacit Knowledge and Troubleshooting, cons@32, gpt-5.2 browsing, printed chart label (Figure 7). Text gives 83.33% with refusals counted as passes Bio and chem risk
Nov 18, 2025 GPT-5.1-Codex-Max system card 77 Biorisk Tacit Knowledge and Troubleshooting, cons@32, printed chart label and text (82% with refusals counted) (Figure 4, 5.1.1.4) Bio and chem risk
Aug 7, 2025 GPT-5 system card 66 Biorisk Tacit Knowledge and Troubleshooting, cons@32, browsing launch candidate (Figure 12) Bio and chem risk
Aug 7, 2025 GPT-5 mini system card 68 Biorisk Tacit Knowledge and Troubleshooting, cons@32, gpt-5-thinking-mini browsing launch candidate (Figure 12) Bio and chem risk
Aug 5, 2025 gpt-oss-20b model card 73 Biorisk Tacit Knowledge and Troubleshooting, cons@32, gpt-oss 20b browsing launch candidate (Figure 8) Bio and chem risk
Aug 5, 2025 gpt-oss-120b model card 74 Biorisk Tacit Knowledge and Troubleshooting, cons@32, gpt-oss 120b browsing launch candidate (Figure 8) Bio and chem risk
Jul 17, 2025 ChatGPT agent system card Biorisk Tacit Knowledge and Troubleshooting, cons@32, chart shows only ChatGPT Agent browsing helpful-only at 74%, no released-model value (Figure 4, 5.1.1.4) Bio and chem risk
Apr 16, 2025 o4-mini system card 68 Biorisk Tacit Knowledge and Troubleshooting, cons@32, o4-mini no browsing launch candidate, printed chart label (Figure 6) Bio and chem risk
Apr 16, 2025 o3 system card 70 Biorisk Tacit Knowledge and Troubleshooting, cons@32, o3 no browsing launch candidate, printed chart label (Figure 6) Bio and chem risk
Feb 27, 2025 GPT-4.5 system card 72 Tacit knowledge and troubleshooting, post-mitigation (4.3.5) Bio and chem risk
Feb 25, 2025 OpenAI deep research system card 72 Biorisk Tacit Knowledge and Troubleshooting, cons@32, post-mitigation with browsing, printed chart label (without browsing 68%) (3.4.4) Bio and chem risk
Jan 31, 2025 o3-mini system card 58 Biorisk Tacit Knowledge and Troubleshooting, cons@32, o3-mini (Post), printed chart label (5.4.8) Bio and chem risk
Dec 5, 2024 o1 system card 66 Biorisk Tacit Knowledge and Troubleshooting, cons@32, o1 (post-mitigation), printed chart label (5.5.8) Bio and chem risk
Sep 12, 2024 o1-preview system card 66.7 Biorisk Tacit Knowledge and Troubleshooting, pass rate cons@32, o1-preview (post-mitigation), printed chart label (4.3.6) Bio and chem risk
Sep 12, 2024 o1-mini system card 66.7 Biorisk Tacit Knowledge and Troubleshooting, pass rate cons@32, o1-mini (post-mitigation), printed chart label (4.3.6) Bio and chem risk
Aug 8, 2024 GPT-4o system card 69 Tacit knowledge and troubleshooting evaluation set, consensus@10 (3.6) Bio and chem risk

More in Biology and chemistry

All 108 in Biology and chemistry
  • GeneBench Pro In use Agents analyze messy simulated genomics and biology datasets to estimate a decision-relevant quantity across 129 problems. Scored automatically against known answers. GPT-5.6 Sol 31.5%
  • SUPERChem In use Models solve 500 expert-written chemistry reasoning problems, offered in multimodal and text-only forms. Expert solution paths also allow a reasoning path score. Tencent Hy4 preview 66.4%
  • LifeSciBench In use Models complete 750 expert-written life science research tasks, often with attached figures, tables, or sequence files. Graded against expert rubrics as score and pass rate. GPT-Rosalind-6 61.3%
  • MedChemBench (OpenAI) In use Models answer chemistry questions from an OpenAI internal set, grouped with OpenAI's life science research evaluations. OpenAI has not published the task format. GPT-Rosalind-6 49.6%
  • ProtocolQA Open-Ended In use Models correct errors in published lab protocols using open-ended questions adapted from LAB-Bench ProtocolQA. Scored as pass@1. GPT-6 Cyber 47.2%
  • MolecularIQ In use Models answer questions about molecular graphs, such as counting, indexing and constrained generation, and a symbolic checker verifies each answer. gpt-oss-120b 47.5%