Sources
88 documents. Open one for the numbers it supports, each with its quote and page.
Blog post
Introducing HealthAdminBench: AI Agents Can Diagnose Rare Diseases, But Can They Handle Your Insurance? (Kinetic Systems blog)Kinetic Systems, third-party, 14 Apr 20265 results
Open the document kineticsystems.ai, retrieved 7 Sep 2026
Board Row Printed Where Role HealthAdminBench Claude Opus 4.6 (computer-use agent) Quote
In our most realistic set-up (screenshot only observation, detailed prompting), the best-performing agent, Claude Opus 4.6 CUA, achieved only a 36.3% task success rate.
36.3% Section 'LLMs struggle with long-horizon tasks' Also prints it HealthAdminBench GPT-5.4 (computer-use agent) Quote
GPT-5.4 CUA came in a distant second at 26.7%, followed by the open-source Kimi K2.5 model at 15.6%, then Qwen 3.5 at 13.3%, and finally Gemini 3.1 Pro at 11.9%.
26.7% Section 'LLMs struggle with long-horizon tasks' Also prints it HealthAdminBench Kimi K2.5 Quote
GPT-5.4 CUA came in a distant second at 26.7%, followed by the open-source Kimi K2.5 model at 15.6%, then Qwen 3.5 at 13.3%, and finally Gemini 3.1 Pro at 11.9%.
15.6% Section 'LLMs struggle with long-horizon tasks' Also prints it HealthAdminBench Qwen 3.5 Quote
GPT-5.4 CUA came in a distant second at 26.7%, followed by the open-source Kimi K2.5 model at 15.6%, then Qwen 3.5 at 13.3%, and finally Gemini 3.1 Pro at 11.9%.
13.3% Section 'LLMs struggle with long-horizon tasks' Also prints it HealthAdminBench Gemini 3.1 Pro Quote
GPT-5.4 CUA came in a distant second at 26.7%, followed by the open-source Kimi K2.5 model at 15.6%, then Qwen 3.5 at 13.3%, and finally Gemini 3.1 Pro at 11.9%.
11.9% Section 'LLMs struggle with long-horizon tasks' Also prints it
Launch post
Introducing Grok 4.7SpaceXAI, first-party, 21 Sep 20262 results
Open the document x.ai
Board Row Printed Where Role HealthBench Professional Grok 4.7 Quote
Grok 4.7 xHigh Grok 4.6 High GPT-5.6 Sol Max Fable 5.1 Max Clinical reasoningHealthBench Professional 56.7% 48.5% 60.5% 62.1%
56.7% Model Improvements comparison table, Clinical reasoning / HealthBench Professional row; Grok 4.7 column; September 21, 2026. Source of record HealthBench Professional Grok 4.6 Quote
Grok 4.7 xHigh Grok 4.6 High GPT-5.6 Sol Max Fable 5.1 Max Clinical reasoningHealthBench Professional 56.7% 48.5% 60.5% 62.1%
48.5% Model Improvements comparison table, Clinical reasoning / HealthBench Professional row; Grok 4.6 column; September 21, 2026. Source of record GPT-6 Astra: A new generation of intelligenceOpenAI, first-party, 3 Sep 20261 result
Open the document openai.com, retrieved 8 Sep 2026
Board Row Printed Where Role HealthBench Professional GPT-6 Astra Quote
| HealthBench Professional (length-adjusted) | 63.4% | 60.5% | 58.1% | 60.9% | 56.4% | 52.1% |
63.4% Science and Health table Also prints it Qwen3.8-Max: A New Bar for Coding and CoworkAlibaba, first-party, 2 Aug 20265 results
Open the document qwen.ai, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) GPT-5.6 Sol Quote
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
81.5 Full Benchmark Table, second (multimodal) table, row MedXpertQA-MM; columns Opus4.8 / Fable5 / Gemini3.1-Pro / GPT5.6-Sol / Qwen3.7-Plus / Qwen3.8-Max Source of record MedXpertQA (MM) Qwen3.8 Max Quote
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
80.4% Multimodal Benchmarks table, Multimodal Reasoning section, row MedXpertQA-MM, column Qwen3.8-Max; header row: | | Opus4.8 | Fable5 | Gemini3.1-Pro | GPT5.6-Sol | Qwen3.7-Plus | Qwen3.8-Max | Source of record MedXpertQA (MM) Claude Fable 5 Quote
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
80.0 Full Benchmark Table, second (multimodal) table, row MedXpertQA-MM; columns Opus4.8 / Fable5 / Gemini3.1-Pro / GPT5.6-Sol / Qwen3.7-Plus / Qwen3.8-Max Source of record MedXpertQA (MM) Claude Opus 4.8 Quote
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
71.7 Full Benchmark Table, second (multimodal) table, row MedXpertQA-MM; columns Opus4.8 / Fable5 / Gemini3.1-Pro / GPT5.6-Sol / Qwen3.7-Plus / Qwen3.8-Max Source of record MedXpertQA (MM) Qwen3.7 Plus Quote
| MedXpertQA-MM | 71.7 | 80.0 | 80.7 | 81.5 | 71.0 | 80.4 |
71.0 Qwen3.8 launch post, Multimodal Benchmarks table, row MedXpertQA-MM, column Qwen3.7-Plus (same value carried forward) Also prints it Qwen3.7-Plus: Multimodal Agent IntelligenceAlibaba, first-party, 31 May 20262 results
Open the document qwen.ai, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) Qwen3.7 Plus Quote
| MedXpertQA-MM | 77.3 | 64.4 | 80.7 | 68.7 | 71.0 |
71.0% Multimodal Benchmarks table, Multimodal Reasoning section, row MedXpertQA-MM, column Qwen3.7-Plus; header row: | | GPT-5.4 (xhigh) | Opus-4.6 Max | Gemini-3.1 Pro | Qwen3.6-Plus | Qwen3.7-Plus | Source of record MedXpertQA (MM) Qwen3.6 Plus Quote
| MedXpertQA-MM | 77.3 | 64.4 | 80.7 | 68.7 | 71.0 |
68.7 Multimodal Benchmarks table, row MedXpertQA-MM; columns GPT-5.4 (xhigh) / Opus-4.6 Max / Gemini-3.1 Pro / Qwen3.6-Plus / Qwen3.7-Plus Source of record Introducing Muse Spark: Scaling Towards Personal SuperintelligenceMeta, first-party, 8 Apr 20265 results
Open the document ai.meta.com, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) Gemini 3.1 Pro Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
81.3% Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Gemini 3.1 Pro High; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.' Source of record MedXpertQA (MM) Muse Spark Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
78.4% Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Muse Spark Thinking; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.' Source of record MedXpertQA (MM) GPT-5.4 Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
77.1% Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column GPT 5.4 Xhigh; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.' Source of record MedXpertQA (MM) Grok 4.20 Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
65.8% Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Grok 4.2 Reasoning; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.' Source of record MedXpertQA (MM) Claude Opus 4.6 Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
64.8% Launch-post benchmark table image, HEALTH section, row MedXpertQA (MM), column Opus 4.6 Max; the same table is page 5 of the Eval Methodology PDF; protocol p. 2: 'MedXpertQA Text/Multimodal: ... The multimodal variant contains 2,000 multimodal medical questions with clinical images (X-rays, histology, dermatology, etc.) and 5 answer choices (A-E). For grading, we use gpt-oss-120b to parse the predicted answer letter from free-form text.' Source of record
Mirror
MedXpertQA (MM) Leaderboard & Scores - September 2026 | BenchLM.aibenchlm.ai, third-party, date not stated8 results
Open the document benchlm.ai, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) Gemini 3.1 Pro Quote
1 Gemini 3.1 Pro [/models/gemini-3-1-pro]Google · Closed 81.3%
81.3% BENCHMARK SCORE TABLE (8 MODELS), rank 1; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror MedXpertQA (MM) Qwen3.8 Max Quote
2 Qwen3.8 Max [/models/qwen3-8-max]Alibaba · Open weight 80.4%
80.4% BENCHMARK SCORE TABLE (8 MODELS), rank 2; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror MedXpertQA (MM) Muse Spark Quote
3 Muse Spark [/models/muse-spark]Meta · Closed 78.4%
78.4% BENCHMARK SCORE TABLE (8 MODELS), rank 3; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror MedXpertQA (MM) GPT-5.4 Quote
4 GPT-5.4 [/models/gpt-5-4]OpenAI · Closed 77.1%
77.1% BENCHMARK SCORE TABLE (8 MODELS), rank 4; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror MedXpertQA (MM) Qwen3.7 Plus Quote
5 Qwen3.7 Plus [/models/qwen3-7-plus]Alibaba · Closed 71.0%
71.0% BENCHMARK SCORE TABLE (8 MODELS), rank 5; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror MedXpertQA (MM) Grok 4.20 Quote
6 Grok 4.20 [/models/grok-4-20-beta]xAI · Closed 65.8%
65.8% BENCHMARK SCORE TABLE (8 MODELS), rank 6; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror MedXpertQA (MM) Claude Opus 4.6 Quote
7 Claude Opus 4.6 [/models/claude-opus-4-6]Anthropic · Closed 64.8%
64.8% BENCHMARK SCORE TABLE (8 MODELS), rank 7; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror MedXpertQA (MM) Gemma 4 12B Quote
8 Gemma 4 12B [/models/gemma-4-12b]Google · Open weight 48.7%
48.7% BENCHMARK SCORE TABLE (8 MODELS), rank 8; the page says 'We mirror the published score view for MedXpertQA (MM)' and 'Updated September 4, 2026', citing Meta's Muse Spark Eval Methodology Mirror
Model card
MAI-Thinking-1: Building a Hill-Climbing MachineMicrosoft AI, first-party, 12 Aug 20261 result
Open the document microsoft.ai, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional MAI-Thinking-1 Quote
Model AIR-Bench CyberSec Instruct CyberSec Auto Long Fact Truthful QA HealthBench Prof. MedXpert QA MAI-Thinking-1 88 63 63 98 88 35 43 Sonnet 4.6 88 62 56 98 88 38 49
35 p. 54, Table 12 'Post-trained model evaluation results on various public benchmarks', Health group, column HealthBench Prof.; protocol Appendix K.6 p. 106: HealthBench Professional introduces a length penalty for the primary metric, to correct for a well-observed correlation between lengthy responses and artificially increased LLM-grader scores. For all reported scores, we use the standard GPT-5.4 grader and rubrics provided by OpenAI. Source of record Muse Spark 1.1 Evaluation ReportMeta, first-party, 9 Jul 20262 results
Open the document research.meta.ai, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional Muse Spark 1.1 Quote
Health | HealthBench Professional | 59.3 | 54.1 | 41.6 | 55.8 | 51.8 (Figure 44 image table row; columns Muse Spark 1.1, Muse Spark, Gemini 3.1 Pro (high), Opus 4.8 (max), GPT 5.5 (xhigh))
59.3 p. 101, Figure 44 'General capability benchmark results' (image), row HealthBench Professional, column Muse Spark 1.1; protocol p. 104 (printed 103): HealthBench Pro comprises 525 evaluation data points graded by rubrics. We use GPT-5.4 with low reasoning effort as the grader and report the length-normalized rubric score as done in their paper. Source of record HealthBench Professional Muse Spark Quote
Health | HealthBench Professional | 59.3 | 54.1 | 41.6 | 55.8 | 51.8 (Figure 44 image table row; columns Muse Spark 1.1, Muse Spark, Gemini 3.1 Pro (high), Opus 4.8 (max), GPT 5.5 (xhigh))
54.1 p. 101, Figure 44 (image), row HealthBench Professional, column Muse Spark; protocol p. 104 (printed 103) Source of record Muse Spark Eval MethodologyMeta, first-party, 8 Apr 20265 results
Open the document ai.meta.com, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) Gemini 3.1 Pro Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
81.3 p. 5, benchmark table image, HEALTH section, row MedXpertQA (MM), column Gemini 3.1 Pro High Also prints it MedXpertQA (MM) Muse Spark Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
78.4 p. 5, benchmark table image, HEALTH section, row MedXpertQA (MM), column Muse Spark Thinking Also prints it MedXpertQA (MM) GPT-5.4 Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
77.1 p. 5, benchmark table image, HEALTH section, row MedXpertQA (MM), column GPT 5.4 Xhigh Also prints it MedXpertQA (MM) Grok 4.20 Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
65.8 p. 5, benchmark table image, HEALTH section, row MedXpertQA (MM), column Grok 4.2 Reasoning Also prints it MedXpertQA (MM) Claude Opus 4.6 Quote
MedXpertQA (MM) | Medical Multiple Choice | 78.4 | 64.8 | 81.3 | 77.1 | 65.8 (image table row; columns Muse Spark Thinking, Opus 4.6 Max, Gemini 3.1 Pro High, GPT 5.4 Xhigh, Grok 4.2 Reasoning)
64.8 p. 5, benchmark table image, HEALTH section, row MedXpertQA (MM), column Opus 4.6 Max Also prints it Gemma 4 model cardGoogle, first-party, 2 Apr 20265 results
Open the document ai.google.dev, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) Gemma 4 31B Quote
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
61.3% Evaluation Results, MedXPertQA MM row, Gemma 4 31B column. Source of record MedXpertQA (MM) Gemma 4 26B A4B Quote
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
58.1% Evaluation Results, MedXPertQA MM row, Gemma 4 26B A4B column. Source of record MedXpertQA (MM) Gemma 4 12B Quote
| MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | \- |
48.7% Benchmark Results table, Vision section, row MedXPertQA MM, column Gemma 4 12B Unified; header row: | | Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) | Source of record MedXpertQA (MM) Gemma 4 E4B Quote
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
28.7% Evaluation Results, MedXPertQA MM row, Gemma 4 E4B column. Source of record MedXpertQA (MM) Gemma 4 E2B Quote
| Gemma 4 31B | Gemma 4 26B A4B | Gemma 4 12B Unified | Gemma 4 E4B | Gemma 4 E2B | Gemma 3 27B (no think) MedXPertQA MM | 61.3% | 58.1% | 48.7% | 28.7% | 23.5% | -
23.5% Evaluation Results, MedXPertQA MM row, Gemma 4 E2B column. Source of record Qwen/Qwen3.5-397B-A17B model cardAlibaba / Qwen, first-party, 16 Feb 20266 results
Open the document huggingface.co, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) Gemini-3 Pro Quote
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
76.0 Benchmark Results > Vision Language > Medical VQA; MedXpertQA-MM row, Gemini-3 Pro column. Source of record MedXpertQA (MM) GPT-5.2 Quote
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
73.3 Benchmark Results > Vision Language > Medical VQA, row MedXpertQA-MM; columns GPT5.2 / Claude 4.5 Opus / Gemini-3 Pro / Qwen3-VL-235B-A22B / K2.5-1T-A32B / Qwen3.5-397B-A17B Source of record MedXpertQA (MM) Qwen3.5 397B A17B Quote
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
70.0 Benchmark Results > Vision Language > Medical VQA, row MedXpertQA-MM; columns GPT5.2 / Claude 4.5 Opus / Gemini-3 Pro / Qwen3-VL-235B-A22B / K2.5-1T-A32B / Qwen3.5-397B-A17B Source of record MedXpertQA (MM) Kimi K2.5 Quote
MedXpertQA-MM 73.3 63.6 76.0 47.6 65.3 70.0
65.3 Benchmark Results > Vision Language > Medical VQA, row MedXpertQA-MM; columns GPT5.2 / Claude 4.5 Opus / Gemini-3 Pro / Qwen3-VL-235B-A22B / K2.5-1T-A32B / Qwen3.5-397B-A17B Source of record MedXpertQA (MM) Claude 4.5 Opus Quote
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
63.6 Benchmark Results > Vision Language > Medical VQA; MedXpertQA-MM row, Claude 4.5 Opus column. Source of record MedXpertQA (MM) Qwen3-VL-235B-A22B Quote
| GPT5.2 | Claude 4.5 Opus | Gemini-3 Pro | Qwen3-VL-235B-A22B | K2.5-1T-A32B | Qwen3.5-397B-A17B MedXpertQA-MM | 73.3 | 63.6 | 76.0 | 47.6 | 65.3 | 70.0
47.6 Benchmark Results > Vision Language > Medical VQA; MedXpertQA-MM row, Qwen3-VL-235B-A22B column. Source of record
Official leaderboard
Vals AI MedCode leaderboardVals AI, first-party, 29 Sep 2026103 results
Open the document www.vals.ai, retrieved 8 Sep 2026
Board Row Printed Where Role MedCode (Vals AI) Claude Opus 5 Quote
1 | Claude Opus 5 | 63.57%±1.99 | $5/$25 | 24.50s
63.57% MedCode leaderboard, Overall task, All Models expanded, rank 1 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-5"]. Source of record MedCode (Vals AI) Gemini 3.1 Pro Preview (02/26) Quote
2 | Gemini 3.1 Pro Preview (02/26) | 59.06%±2.00 | $2/$12 | 38.52s
59.06% MedCode leaderboard, Overall task, All Models expanded, rank 2 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3.1-pro-preview"]. Source of record MedCode (Vals AI) Claude Fable 5 Quote
3 | Claude Fable 5 | 56.07%±2.20 | $10/$50 | 91.44s
56.07% MedCode leaderboard, Overall task, All Models expanded, rank 3 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-fable-5"]. Source of record MedCode (Vals AI) Gemini 3 Flash (12/25) Quote
4 | Gemini 3 Flash (12/25) | 55.92%±2.11 | $0.5/$3 | 44.15s
55.92% MedCode leaderboard, Overall task, All Models expanded, rank 4 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3-flash-preview"]. Source of record MedCode (Vals AI) Gemini 3.5 Flash Quote
5 | Gemini 3.5 Flash | 55.83%±2.11 | $1.5/$9 | 25.29s
55.83% MedCode leaderboard, Overall task, All Models expanded, rank 5 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3.5-flash"]. Source of record MedCode (Vals AI) Claude Opus 4.7 Quote
6 | Claude Opus 4.7 | 54.86%±2.21 | $5/$25 | 54.25s
54.86% MedCode leaderboard, Overall task, All Models expanded, rank 6 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-7"]. Source of record MedCode (Vals AI) Claude Fable 5.1 Quote
7 | Claude Fable 5.1 | 53.51%±2.17 | $10/$50 | 3m35s
53.51% MedCode leaderboard, Overall task, All Models expanded, rank 7 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-fable-5-1"]. Source of record MedCode (Vals AI) Gemini 3.7 Flash Quote
8 | Gemini 3.7 Flash | 53.39%±2.12 | $1.5/$7.5 | 9.16s
53.39% MedCode leaderboard, Overall task, All Models expanded, rank 8 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3.7-flash"]. Source of record MedCode (Vals AI) Claude Opus 4.8 Quote
9 | Claude Opus 4.8 | 53.22%±2.17 | $5/$25 | 105.92s
53.22% MedCode leaderboard, Overall task, All Models expanded, rank 9 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-8"]. Source of record MedCode (Vals AI) Gemini 3.6 Flash Quote
10 | Gemini 3.6 Flash | 53.15%±2.16 | $1.5/$7.5 | 16.68s
53.15% MedCode leaderboard, Overall task, All Models expanded, rank 10 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3.6-flash"]. Source of record MedCode (Vals AI) Claude Sonnet 5.5 Quote
11 | Claude Sonnet 5.5 | 52.92%±2.12 | $2/$10 | 3m59s
52.92% MedCode leaderboard, Overall task, All Models expanded, rank 11 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-sonnet-5-5"]. Source of record MedCode (Vals AI) GPT 5.1 Quote
12 | GPT 5.1 | 52.73%±2.15 | $1.25/$10 | 54.55s
52.73% MedCode leaderboard, Overall task, All Models expanded, rank 12 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.1-2025-11-13"]. Source of record MedCode (Vals AI) Gemini 3 Pro (11/25) Quote
13 | Gemini 3 Pro (11/25) | 52.20%±2.07 | $2/$12 | 55.84s
52.20% MedCode leaderboard, Overall task, All Models expanded, rank 13 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3-pro-preview"]. Source of record MedCode (Vals AI) Muse Spark Quote
14 | Muse Spark | 51.31%±2.24 | N/A | 2m03s
51.31% MedCode leaderboard, Overall task, All Models expanded, rank 14 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["meta/muse_spark"]. Source of record MedCode (Vals AI) Gemini 2.5 Pro Quote
15 | Gemini 2.5 Pro | 50.59%±2.11 | $1.25/$10 | 27.41s
50.59% MedCode leaderboard, Overall task, All Models expanded, rank 15 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-pro"]. Source of record MedCode (Vals AI) Claude Opus 5.5 Quote
16 | Claude Opus 5.5 | 49.80%±2.27 | $4/$20 | 4m07s
49.80% MedCode leaderboard, Overall task, All Models expanded, rank 16 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-5-5"]. Source of record MedCode (Vals AI) GPT 5.2 Quote
17 | GPT 5.2 | 49.75%±2.26 | $1.75/$14 | 2m37s
49.75% MedCode leaderboard, Overall task, All Models expanded, rank 17 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.2-2025-12-11"]. Source of record MedCode (Vals AI) GPT 5 Quote
18 | GPT 5 | 49.63%±2.10 | $1.25/$10 | 58.27s
49.63% MedCode leaderboard, Overall task, All Models expanded, rank 18 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5-2025-08-07"]. Source of record MedCode (Vals AI) Grok 4.7 Quote
19 | Grok 4.7 | 49.55%±2.17 | $2/$6 | 3m33s
49.55% MedCode leaderboard, Overall task, All Models expanded, rank 19 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4.7"]. Source of record MedCode (Vals AI) Muse Spark 1.2 Quote
20 | Muse Spark 1.2 | 49.35%±2.19 | $1.25/$4.25 | 60.17s
49.35% MedCode leaderboard, Overall task, All Models expanded, rank 20 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["meta/muse_spark_1_2"]. Source of record MedCode (Vals AI) Claude Opus 4.5 (Thinking) Quote
21 | Claude Opus 4.5 (Thinking) | 49.16%±2.01 | $5/$25 | 60.84s
49.16% MedCode leaderboard, Overall task, All Models expanded, rank 21 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-5-20251101-thinking"]. Source of record MedCode (Vals AI) Claude Opus 4.6 (Thinking) Quote
22 | Claude Opus 4.6 (Thinking) | 49.13%±2.08 | $5/$25 | 2m36s
49.13% MedCode leaderboard, Overall task, All Models expanded, rank 22 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-6-thinking"]. Source of record MedCode (Vals AI) GPT 5.5 Quote
23 | GPT 5.5 | 49.10%±2.19 | $5/$30 | 2m40s
49.10% MedCode leaderboard, Overall task, All Models expanded, rank 23 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.5"]. Source of record MedCode (Vals AI) Kimi K3 Quote
24 | Kimi K3 | 48.88%±2.19 | $3/$15 | 116.18s
48.88% MedCode leaderboard, Overall task, All Models expanded, rank 24 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["kimi/kimi-k3"]. Source of record MedCode (Vals AI) GPT-6.1 Sol Quote
25 | GPT-6.1 Sol | 48.84%±2.12 | $2/$10 | 115.34s
48.84% MedCode leaderboard, Overall task, All Models expanded, rank 25 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-6.1-sol"]. Source of record MedCode (Vals AI) GPT-6 Astra Quote
26 | GPT-6 Astra | 48.49%±2.13 | $10/$50 | 102.86s
48.49% MedCode leaderboard, Overall task, All Models expanded, rank 26 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-6-astra"]. Source of record MedCode (Vals AI) Claude Opus 4.6 (Nonthinking) Quote
27 | Claude Opus 4.6 (Nonthinking) | 48.24%±2.05 | $5/$25 | 4.06s
48.24% MedCode leaderboard, Overall task, All Models expanded, rank 27 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-6"]. Source of record MedCode (Vals AI) Gemini 3.8 Flash Quote
28 | Gemini 3.8 Flash | 48.13%±2.18 | $1.5/$7.5 | 42.89s
48.13% MedCode leaderboard, Overall task, All Models expanded, rank 28 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3.8-flash"]. Source of record MedCode (Vals AI) Gemini 3.1 Flash Lite Preview Quote
29 | Gemini 3.1 Flash Lite Preview | 47.60%±2.07 | $0.25/$1.5 | 8.55s
47.60% MedCode leaderboard, Overall task, All Models expanded, rank 29 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3.1-flash-lite-preview"]. Source of record MedCode (Vals AI) Claude Sonnet 5 Quote
30 | Claude Sonnet 5 | 47.54%±2.27 | $2/$10 | 2m15s
47.54% MedCode leaderboard, Overall task, All Models expanded, rank 30 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-sonnet-5"]. Source of record MedCode (Vals AI) o3 Quote
31 | o3 | 47.29%±2.16 | $2/$8 | 17.68s
47.29% MedCode leaderboard, Overall task, All Models expanded, rank 31 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/o3-2025-04-16"]. Source of record MedCode (Vals AI) Claude Opus 4.1 (Thinking) Quote
32 | Claude Opus 4.1 (Thinking) | 47.23%±2.07 | $15/$75 | 33.26s
47.23% MedCode leaderboard, Overall task, All Models expanded, rank 32 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-1-20250805-thinking"]. Source of record MedCode (Vals AI) GPT-6 Sol Quote
33 | GPT-6 Sol | 47.07%±2.12 | $2/$10 | 62.71s
47.07% MedCode leaderboard, Overall task, All Models expanded, rank 33 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-6-sol"]. Source of record MedCode (Vals AI) MiniMax-M3 Quote
34 | MiniMax-M3 | 46.29%±2.10 | $0.6/$2.4 | 63.12s
46.29% MedCode leaderboard, Overall task, All Models expanded, rank 34 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["minimax/MiniMax-M3"]. Source of record MedCode (Vals AI) Claude Opus 4.5 (Nonthinking) Quote
35 | Claude Opus 4.5 (Nonthinking) | 45.17%±1.89 | $5/$25 | 5.02s
45.17% MedCode leaderboard, Overall task, All Models expanded, rank 35 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-5-20251101"]. Source of record MedCode (Vals AI) MiMo V2.6 Pro Quote
36 | MiMo V2.6 Pro | 44.97%±2.10 | $0.435/$0.87 | 2m57s
44.97% MedCode leaderboard, Overall task, All Models expanded, rank 36 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["xiaomi/mimo-v2.6-pro"]. Source of record MedCode (Vals AI) Grok 4.6 Quote
37 | Grok 4.6 | 44.71%±2.26 | $2/$6 | 2m07s
44.71% MedCode leaderboard, Overall task, All Models expanded, rank 37 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4.6"]. Source of record MedCode (Vals AI) GPT-6 Luna Quote
38 | GPT-6 Luna | 44.69%±2.30 | $0.1/$0.5 | 108.77s
44.69% MedCode leaderboard, Overall task, All Models expanded, rank 38 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-6-luna"]. Source of record MedCode (Vals AI) Claude Sonnet 4.5 (Thinking) Quote
39 | Claude Sonnet 4.5 (Thinking) | 44.13%±2.00 | $3/$15 | 74.33s
44.13% MedCode leaderboard, Overall task, All Models expanded, rank 39 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-5-20250929-thinking"]. Source of record MedCode (Vals AI) GPT-5.6 Sol Quote
40 | GPT-5.6 Sol | 43.97%±2.26 | $4/$20 | 96.59s
43.97% MedCode leaderboard, Overall task, All Models expanded, rank 40 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.6-sol"]. Source of record MedCode (Vals AI) Gemini 3.5 Flash Lite Quote
41 | Gemini 3.5 Flash Lite | 43.49%±1.95 | $0.3/$2.5 | 6.02s
43.49% MedCode leaderboard, Overall task, All Models expanded, rank 41 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-3.5-flash-lite"]. Source of record MedCode (Vals AI) GPT-5.6 Terra Quote
42 | GPT-5.6 Terra | 43.41%±2.17 | $2/$12 | 18.41s
43.41% MedCode leaderboard, Overall task, All Models expanded, rank 42 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.6-terra"]. Source of record MedCode (Vals AI) Grok 4.5 Quote
43 | Grok 4.5 | 43.29%±2.31 | $2/$6 | 59.64s
43.29% MedCode leaderboard, Overall task, All Models expanded, rank 43 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4.5"]. Source of record MedCode (Vals AI) Hy4 Preview Quote
44 | Hy4 Preview | 43.25%±2.13 | $0.834/$2.501 | 6m39s
43.25% MedCode leaderboard, Overall task, All Models expanded, rank 44 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["tencent/hy4-preview"]. Source of record MedCode (Vals AI) GPT 5 Mini Quote
45 | GPT 5 Mini | 43.05%±2.04 | $0.25/$2 | 28.17s
43.05% MedCode leaderboard, Overall task, All Models expanded, rank 45 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5-mini-2025-08-07"]. Source of record MedCode (Vals AI) GLM 5.3 Quote
46 | GLM 5.3 | 42.86%±2.11 | $1.4/$4.4 | 2m49s
42.86% MedCode leaderboard, Overall task, All Models expanded, rank 46 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["zai/glm-5.3"]. Source of record MedCode (Vals AI) DeepSeek V4 Pro 0813 Quote
47 | DeepSeek V4 Pro 0813 | 42.47%±2.16 | $1.32/$3.96 | 3m01s
42.47% MedCode leaderboard, Overall task, All Models expanded, rank 47 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["deepseek/deepseek-v4-pro-0813"]. Source of record MedCode (Vals AI) GPT-5.6 Luna Quote
48 | GPT-5.6 Luna | 42.39%±2.27 | $0.2/$1.2 | 81.28s
42.39% MedCode leaderboard, Overall task, All Models expanded, rank 48 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.6-luna"]. Source of record MedCode (Vals AI) GLM 5.1 Quote
49 | GLM 5.1 | 41.60%±2.12 | $1/$3.2 | 77.58s
41.60% MedCode leaderboard, Overall task, All Models expanded, rank 49 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["zai/glm-5.1"]. Source of record MedCode (Vals AI) DeepSeek V4 Flash 0731 Quote
50 | DeepSeek V4 Flash 0731 | 41.41%±2.15 | $0.44/$1.32 | 2m07s
41.41% MedCode leaderboard, Overall task, All Models expanded, rank 50 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["deepseek/deepseek-v4-flash-0731"]. Source of record MedCode (Vals AI) Claude Opus 4.1 (Nonthinking) Quote
51 | Claude Opus 4.1 (Nonthinking) | 41.37%±1.96 | $15/$75 | 13.08s
41.37% MedCode leaderboard, Overall task, All Models expanded, rank 51 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-opus-4-1-20250805"]. Source of record MedCode (Vals AI) GPT 5.4 (xhigh) Quote
52 | GPT 5.4 (xhigh) | 41.29%±2.15 | $2.5/$15 | 3m07s
41.29% MedCode leaderboard, Overall task, All Models expanded, rank 52 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.4-2026-03-05"]. Source of record MedCode (Vals AI) Inkling Quote
53 | Inkling | 41.19%±2.23 | $1/$4.05 | 2m47s
41.19% MedCode leaderboard, Overall task, All Models expanded, rank 53 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["thinkingmachines/inkling"]. Source of record MedCode (Vals AI) DeepSeek V4.1 Flash Quote
54 | DeepSeek V4.1 Flash | 41.17%±2.04 | $0.3/$1.2 | 35.13s
41.17% MedCode leaderboard, Overall task, All Models expanded, rank 54 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["deepseek/deepseek-v4.1-flash"]. Source of record MedCode (Vals AI) MiMo V2.6 Flash Quote
55 | MiMo V2.6 Flash | 41.06%±2.03 | $0.14/$0.28 | 93.37s
41.06% MedCode leaderboard, Overall task, All Models expanded, rank 55 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["xiaomi/mimo-v2.6-flash"]. Source of record MedCode (Vals AI) GPT 5.4 Nano Quote
56 | GPT 5.4 Nano | 41.03%±2.26 | $0.2/$1.25 | 8.04s
41.03% MedCode leaderboard, Overall task, All Models expanded, rank 56 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5.4-nano-2026-03-17"]. Source of record MedCode (Vals AI) GLM 5.2 Quote
57 | GLM 5.2 | 40.77%±2.17 | $1.4/$4.4 | 95.27s
40.77% MedCode leaderboard, Overall task, All Models expanded, rank 57 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["zai/glm-5.2"]. Source of record MedCode (Vals AI) Qwen 3.8 Max Quote
58 | Qwen 3.8 Max | 40.67%±2.03 | $2/$6 | 6m18s
40.67% MedCode leaderboard, Overall task, All Models expanded, rank 58 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["alibaba/qwen3.8-max"]. Source of record MedCode (Vals AI) Claude Sonnet 4.5 (Nonthinking) Quote
59 | Claude Sonnet 4.5 (Nonthinking) | 40.57%±2.00 | $3/$15 | 12.01s
40.57% MedCode leaderboard, Overall task, All Models expanded, rank 59 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-5-20250929"]. Source of record MedCode (Vals AI) Gemini 2.5 Flash Preview (9/25) (Nonthinking) Quote
60 | Gemini 2.5 Flash Preview (9/25) (Nonthinking) | 40.54%±1.93 | $0.3/$2.5 | 12.70s
40.54% MedCode leaderboard, Overall task, All Models expanded, rank 60 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-flash-preview-09-2025"]. Source of record MedCode (Vals AI) DeepSeek V4 Quote
61 | DeepSeek V4 | 40.45%±2.12 | $1.32/$3.96 | 6m23s
40.45% MedCode leaderboard, Overall task, All Models expanded, rank 61 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["deepseek/deepseek-v4-pro"]. Source of record MedCode (Vals AI) Gemini 2.5 Flash (7/17) (Thinking) Quote
62 | Gemini 2.5 Flash (7/17) (Thinking) | 40.36%±1.95 | $0.3/$2.5 | 22.83s
40.36% MedCode leaderboard, Overall task, All Models expanded, rank 62 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-flash-thinking"]. Source of record MedCode (Vals AI) Gemini 2.5 Flash Preview (9/25) (Thinking) Quote
63 | Gemini 2.5 Flash Preview (9/25) (Thinking) | 40.33%±1.92 | $0.3/$2.5 | 16.11s
40.33% MedCode leaderboard, Overall task, All Models expanded, rank 63 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-flash-preview-09-2025-thinking"]. Source of record MedCode (Vals AI) Kimi K2.6 Quote
64 | Kimi K2.6 | 40.14%±2.04 | $0.95/$4 | 5m05s
40.14% MedCode leaderboard, Overall task, All Models expanded, rank 64 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["kimi/kimi-k2.6"]. Source of record MedCode (Vals AI) Kimi K2.5 Quote
65 | Kimi K2.5 | 39.32%±2.12 | $0.6/$3 | 75.45s
39.32% MedCode leaderboard, Overall task, All Models expanded, rank 65 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["kimi/kimi-k2.5-thinking"]. Source of record MedCode (Vals AI) Qwen 3.7 Max Quote
66 | Qwen 3.7 Max | 38.75%±2.20 | $2.5/$7.5 | 28.75s
38.75% MedCode leaderboard, Overall task, All Models expanded, rank 66 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["alibaba/qwen3.7-max"]. Source of record MedCode (Vals AI) Nemotron 3 Ultra Quote
67 | Nemotron 3 Ultra | 38.62%±2.00 | N/A | 18.68s
38.62% MedCode leaderboard, Overall task, All Models expanded, rank 67 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["nvidia/nemotron-3-ultra-550b-a55b"]. Source of record MedCode (Vals AI) Gemini 2.5 Flash (7/17) (Nonthinking) Quote
68 | Gemini 2.5 Flash (7/17) (Nonthinking) | 38.42%±1.92 | $0.3/$2.5 | 22.14s
38.42% MedCode leaderboard, Overall task, All Models expanded, rank 68 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-flash"]. Source of record MedCode (Vals AI) Grok 4 Quote
69 | Grok 4 | 38.08%±2.21 | $3/$15 | 89.00s
38.08% MedCode leaderboard, Overall task, All Models expanded, rank 69 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4-0709"]. Source of record MedCode (Vals AI) Grok 4.3 Quote
70 | Grok 4.3 | 38.07%±2.08 | $1.25/$2.5 | 43.92s
38.07% MedCode leaderboard, Overall task, All Models expanded, rank 70 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4.3"]. Source of record MedCode (Vals AI) Inkling Small Quote
71 | Inkling Small | 37.89%±2.21 | $0.3/$1.2 | 3m11s
37.89% MedCode leaderboard, Overall task, All Models expanded, rank 71 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["thinkingmachines/inkling-small"]. Source of record MedCode (Vals AI) Grok 4 Fast (Reasoning) Quote
72 | Grok 4 Fast (Reasoning) | 37.38%±1.94 | $0.2/$0.5 | 17.52s
37.38% MedCode leaderboard, Overall task, All Models expanded, rank 72 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4-fast-reasoning"]. Source of record MedCode (Vals AI) Qwen 3.6 Plus Quote
73 | Qwen 3.6 Plus | 36.89%±2.02 | $0.5/$3 | 56.22s
36.89% MedCode leaderboard, Overall task, All Models expanded, rank 73 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["alibaba/qwen3.6-plus"]. Source of record MedCode (Vals AI) Llama 4 Maverick Quote
74 | Llama 4 Maverick | 36.51%±1.99 | $0.22/$0.88 | 21.24s
36.51% MedCode leaderboard, Overall task, All Models expanded, rank 74 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["fireworks/llama4-maverick-instruct-basic"]. Source of record MedCode (Vals AI) Claude Sonnet 4 (Thinking) Quote
75 | Claude Sonnet 4 (Thinking) | 34.96%±1.94 | $3/$15 | 39.80s
34.96% MedCode leaderboard, Overall task, All Models expanded, rank 75 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-20250514-thinking"]. Source of record MedCode (Vals AI) MiniMax-M2.7 Quote
76 | MiniMax-M2.7 | 34.44%±1.99 | $0.3/$1.2 | 30.25s
34.44% MedCode leaderboard, Overall task, All Models expanded, rank 76 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["minimax/MiniMax-M2.7"]. Source of record MedCode (Vals AI) Gemini 2.5 Flash Lite (9/25) (Thinking) Quote
77 | Gemini 2.5 Flash Lite (9/25) (Thinking) | 34.19%±1.74 | $0.1/$0.4 | 10.49s
34.19% MedCode leaderboard, Overall task, All Models expanded, rank 77 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-flash-lite-preview-09-2025-thinking"]. Source of record MedCode (Vals AI) MiniMax-M2.1 Quote
78 | MiniMax-M2.1 | 34.08%±1.94 | $0.3/$1.2 | 19.55s
34.08% MedCode leaderboard, Overall task, All Models expanded, rank 78 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["minimax/MiniMax-M2.1"]. Source of record MedCode (Vals AI) Claude Sonnet 4 (Nonthinking) Quote
79 | Claude Sonnet 4 (Nonthinking) | 33.94%±1.91 | $3/$15 | 7.30s
33.94% MedCode leaderboard, Overall task, All Models expanded, rank 79 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-20250514"]. Source of record MedCode (Vals AI) o4 Mini Quote
80 | o4 Mini | 33.79%±2.02 | $1.1/$4.4 | 21.06s
33.79% MedCode leaderboard, Overall task, All Models expanded, rank 80 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/o4-mini-2025-04-16"]. Source of record MedCode (Vals AI) Mistral Medium 3.5 Quote
81 | Mistral Medium 3.5 | 33.75%±1.15 | $1.5/$7.5 | 34.88s
33.75% MedCode leaderboard, Overall task, All Models expanded, rank 81 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["mistralai/mistral-medium-3.5"]. Source of record MedCode (Vals AI) Qwen 3.5 Flash Quote
82 | Qwen 3.5 Flash | 33.00%±1.79 | $0.1/$0.4 | 63.07s
33.00% MedCode leaderboard, Overall task, All Models expanded, rank 82 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["alibaba/qwen3.5-flash"]. Source of record MedCode (Vals AI) GLM 4.7 Quote
83 | GLM 4.7 | 32.77%±2.00 | $0.6/$2.2 | 2m03s
32.77% MedCode leaderboard, Overall task, All Models expanded, rank 83 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["zai/glm-4.7"]. Source of record MedCode (Vals AI) Claude Haiku 4.5 (Thinking) Quote
84 | Claude Haiku 4.5 (Thinking) | 32.68%±2.00 | $1/$5 | 34.29s
32.68% MedCode leaderboard, Overall task, All Models expanded, rank 84 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["anthropic/claude-haiku-4-5-20251001-thinking"]. Source of record MedCode (Vals AI) MiMo V2.5 Pro Quote
85 | MiMo V2.5 Pro | 32.48%±1.91 | $0.435/$0.87 | 38.45s
32.48% MedCode leaderboard, Overall task, All Models expanded, rank 85 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["xiaomi/mimo-v2.5-pro"]. Source of record MedCode (Vals AI) Ling 3.0 Flash Quote
86 | Ling 3.0 Flash | 32.27%±1.91 | $0.075/$0.22 | 9.65s
32.27% MedCode leaderboard, Overall task, All Models expanded, rank 86 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["ant/ling-3.0-flash-2607"]. Source of record MedCode (Vals AI) Grok 4.20 (Reasoning) Quote
87 | Grok 4.20 (Reasoning) | 32.16%±2.12 | $2/$6 | 16.55s
32.16% MedCode leaderboard, Overall task, All Models expanded, rank 87 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4.20-0309-reasoning"]. Source of record MedCode (Vals AI) MiMo V2.5 Quote
88 | MiMo V2.5 | 31.89%±2.02 | $0.14/$0.28 | 17.70s
31.89% MedCode leaderboard, Overall task, All Models expanded, rank 88 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["xiaomi/mimo-v2.5"]. Source of record MedCode (Vals AI) Qwen 3 VL Plus Quote
89 | Qwen 3 VL Plus | 31.65%±1.84 | $0.2/$1.6 | 9.95s
31.65% MedCode leaderboard, Overall task, All Models expanded, rank 89 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["alibaba/qwen3-vl-plus-2025-09-23"]. Source of record MedCode (Vals AI) Qwen 3 Max Thinking Quote
90 | Qwen 3 Max Thinking | 31.37%±1.89 | $1.2/$6 | 3m02s
31.37% MedCode leaderboard, Overall task, All Models expanded, rank 90 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["alibaba/qwen3-max-2026-01-23"]. Source of record MedCode (Vals AI) Mercury 2.5 Quote
91 | Mercury 2.5 | 31.33%±1.95 | $0.2/$0.75 | 7.93s
31.33% MedCode leaderboard, Overall task, All Models expanded, rank 91 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["inception/mercury-2.5"]. Source of record MedCode (Vals AI) GPT 5 Nano Quote
92 | GPT 5 Nano | 30.44%±1.95 | $0.05/$0.4 | 29.74s
30.44% MedCode leaderboard, Overall task, All Models expanded, rank 92 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["openai/gpt-5-nano-2025-08-07"]. Source of record MedCode (Vals AI) Grok 4 Fast (Non-Reasoning) Quote
93 | Grok 4 Fast (Non-Reasoning) | 30.04%±1.97 | $0.2/$0.5 | 18.21s
30.04% MedCode leaderboard, Overall task, All Models expanded, rank 93 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4-fast-non-reasoning"]. Source of record MedCode (Vals AI) Ling 3.0 Flash Fin Quote
94 | Ling 3.0 Flash Fin | 29.30%±1.94 | $0.06/$0.18 | 31.17s
29.30% MedCode leaderboard, Overall task, All Models expanded, rank 94 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["ant/ling-3.0-flash-af-rc3"]. Source of record MedCode (Vals AI) Qwen 3.8 27B Quote
95 | Qwen 3.8 27B | 28.70%±1.97 | $0.5/$3 | 89.38s
28.70% MedCode leaderboard, Overall task, All Models expanded, rank 95 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["alibaba/qwen3.8-27b"]. Source of record MedCode (Vals AI) Grok 4.1 Fast Non-Reasoning Quote
96 | Grok 4.1 Fast Non-Reasoning | 28.35%±1.92 | $0.2/$0.5 | 4.50s
28.35% MedCode leaderboard, Overall task, All Models expanded, rank 96 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4-1-fast-non-reasoning"]. Source of record MedCode (Vals AI) Grok 4.1 Fast (Reasoning) Quote
97 | Grok 4.1 Fast (Reasoning) | 28.08%±1.99 | $0.2/$0.5 | 46.50s
28.08% MedCode leaderboard, Overall task, All Models expanded, rank 97 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["grok/grok-4-1-fast-reasoning"]. Source of record MedCode (Vals AI) Gemini 2.5 Flash Lite (Nonthinking) Quote
98 | Gemini 2.5 Flash Lite (Nonthinking) | 27.11%±1.84 | $0.1/$0.4 | 5.75s
27.11% MedCode leaderboard, Overall task, All Models expanded, rank 98 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-flash-lite"]. Source of record MedCode (Vals AI) Gemini 2.5 Flash Lite (9/25) (Nonthinking) Quote
99 | Gemini 2.5 Flash Lite (9/25) (Nonthinking) | 27.08%±1.91 | $0.1/$0.4 | 6.24s
27.08% MedCode leaderboard, Overall task, All Models expanded, rank 99 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["google/gemini-2.5-flash-lite-preview-09-2025"]. Source of record MedCode (Vals AI) Llama 4 Scout Quote
100 | Llama 4 Scout | 23.31%±1.75 | $0.18/$0.59 | 10.74s
23.31% MedCode leaderboard, Overall task, All Models expanded, rank 100 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["together/meta-llama/Llama-4-Scout-17B-16E-Instruct"]. Source of record MedCode (Vals AI) Laguna M.1 Quote
101 | Laguna M.1 | 23.11%±1.69 | N/A | 71.00s
23.11% MedCode leaderboard, Overall task, All Models expanded, rank 101 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["poolside/laguna-m.1"]. Source of record MedCode (Vals AI) Laguna XS.2 Quote
102 | Laguna XS.2 | 21.25%±1.70 | N/A | 33.09s
21.25% MedCode leaderboard, Overall task, All Models expanded, rank 102 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["poolside/laguna-xs.2"]. Source of record MedCode (Vals AI) Command A+ Quote
103 | Command A+ | 19.72%±1.83 | N/A | 103.75s
19.72% MedCode leaderboard, Overall task, All Models expanded, rank 103 of 103; Accuracy column; board Updated 9/29/2026. Rendered table row; configuration from embedded BenchmarkView props benchmarkView.default.tasks.overall["cohere/command-a-plus-05-2026"]. Source of record Vals AI MedScribe leaderboardVals AI, first-party, 3 Sep 2026105 results
Open the document www.vals.ai, retrieved 8 Sep 2026
Board Row Printed Where Role MedScribe (Vals AI) Claude Opus 5.5 Quote
1 | Claude Opus 5.5 | 91.43%±1.93 | $4/$20 | 7m22s
91.43% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 1 of 105 (Claude Opus 5.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-5-5"]. Source of record MedScribe (Vals AI) Claude Fable 5.1 Quote
2 | Claude Fable 5.1 | 91.29%±1.95 | $10/$50 | 3m08s
91.29% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 2 of 105 (Claude Fable 5.1), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-fable-5-1"]. Source of record MedScribe (Vals AI) Claude Sonnet 5.5 Quote
3 | Claude Sonnet 5.5 | 91.10%±1.96 | $2/$10 | 5m07s
91.10% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 3 of 105 (Claude Sonnet 5.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-sonnet-5-5"]. Source of record MedScribe (Vals AI) Claude Opus 5 Quote
4 | Claude Opus 5 | 90.98%±1.92 | $5/$25 | 76.56s
90.98% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 4 of 105 (Claude Opus 5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-5"]. Source of record MedScribe (Vals AI) Muse Spark 1.2 Quote
5 | Muse Spark 1.2 | 90.06%±1.96 | $1.25/$4.25 | 61.46s
90.06% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 5 of 105 (Muse Spark 1.2), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["meta/muse_spark_1_2"]. Source of record MedScribe (Vals AI) Grok 4.7 Quote
6 | Grok 4.7 | 89.38%±1.89 | $2/$6 | 2m33s
89.38% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 6 of 105 (Grok 4.7), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4.7"]. Source of record MedScribe (Vals AI) GLM 5.3 Flash Quote
7 | GLM 5.3 Flash | 88.94%±1.91 | $0.075/$0.25 | 86.98s
88.94% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 7 of 105 (GLM 5.3 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["zai/glm-5.3-flash"]. Source of record MedScribe (Vals AI) Muse Spark 1.1 Quote
8 | Muse Spark 1.1 | 88.89%±1.95 | $1.25/$4.25 | 63.34s
88.89% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 8 of 105 (Muse Spark 1.1), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["meta/muse_spark_1_1"]. Source of record MedScribe (Vals AI) GLM 5.3 Quote
9 | GLM 5.3 | 88.81%±2.00 | $1.4/$4.4 | 2m02s
88.81% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 9 of 105 (GLM 5.3), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["zai/glm-5.3"]. Source of record MedScribe (Vals AI) Claude Fable 5 Quote
10 | Claude Fable 5 | 88.52%±1.95 | $10/$50 | 119.47s
88.52% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 10 of 105 (Claude Fable 5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-fable-5"]. Source of record MedScribe (Vals AI) MiMo V2.6 Pro Quote
11 | MiMo V2.6 Pro | 88.31%±1.94 | $0.435/$0.87 | 2m53s
88.31% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 11 of 105 (MiMo V2.6 Pro), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["xiaomi/mimo-v2.6-pro"]. Source of record MedScribe (Vals AI) GPT 5.1 Quote
12 | GPT 5.1 | 88.09%±1.94 | $1.25/$10 | 77.98s
88.09% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 12 of 105 (GPT 5.1), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.1-2025-11-13"]. Source of record MedScribe (Vals AI) Kimi K3 Quote
13 | Kimi K3 | 87.96%±1.89 | $3/$15 | 2m17s
87.96% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 13 of 105 (Kimi K3), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["kimi/kimi-k3"]. Source of record MedScribe (Vals AI) GPT-6 Astra Quote
14 | GPT-6 Astra | 87.91%±1.94 | $10/$50 | 2m34s
87.91% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 14 of 105 (GPT-6 Astra), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-6-astra"]. Source of record MedScribe (Vals AI) MiniMax-M3 Quote
15 | MiniMax-M3 | 87.25%±1.96 | $0.6/$2.4 | 2m04s
87.25% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 15 of 105 (MiniMax-M3), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["minimax/MiniMax-M3"]. Source of record MedScribe (Vals AI) Grok 4.5 Quote
16 | Grok 4.5 | 86.88%±1.94 | $2/$6 | 11m45s
86.88% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 16 of 105 (Grok 4.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4.5"]. Source of record MedScribe (Vals AI) GPT 5.5 Quote
17 | GPT 5.5 | 86.87%±1.93 | $5/$30 | 2m13s
86.87% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 17 of 105 (GPT 5.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.5"]. Source of record MedScribe (Vals AI) Claude Opus 4.6 (Nonthinking) Quote
18 | Claude Opus 4.6 (Nonthinking) | 86.74%±1.94 | $5/$25 | 54.32s
86.74% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 18 of 105 (Claude Opus 4.6 (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-6"]. Source of record MedScribe (Vals AI) Grok 4.6 Quote
19 | Grok 4.6 | 86.53%±1.96 | $2/$6 | 70.94s
86.53% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 19 of 105 (Grok 4.6), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4.6"]. Source of record MedScribe (Vals AI) GPT-6.1 Sol Quote
20 | GPT-6.1 Sol | 86.45%±1.90 | $2/$10 | 2m50s
86.45% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 20 of 105 (GPT-6.1 Sol), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-6.1-sol"]. Source of record MedScribe (Vals AI) Claude Opus 4.6 (Thinking) Quote
21 | Claude Opus 4.6 (Thinking) | 86.13%±1.94 | $5/$25 | 2m08s
86.13% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 21 of 105 (Claude Opus 4.6 (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-6-thinking"]. Source of record MedScribe (Vals AI) Muse Spark Quote
22 | Muse Spark | 85.90%±1.85 | N/A | 3m11s
85.90% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 22 of 105 (Muse Spark), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["meta/muse_spark"]. Source of record MedScribe (Vals AI) Claude Opus 4.8 Quote
23 | Claude Opus 4.8 | 85.75%±1.93 | $5/$25 | 82.36s
85.75% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 23 of 105 (Claude Opus 4.8), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-8"]. Source of record MedScribe (Vals AI) DeepSeek V4.1 Flash Quote
24 | DeepSeek V4.1 Flash | 85.50%±1.92 | $0.3/$1.2 | 53.83s
85.50% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 24 of 105 (DeepSeek V4.1 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["deepseek/deepseek-v4.1-flash"]. Source of record MedScribe (Vals AI) Inkling Quote
25 | Inkling | 85.41%±1.84 | $1/$4.05 | 4m05s
85.41% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 25 of 105 (Inkling), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["thinkingmachines/inkling"]. Source of record MedScribe (Vals AI) Claude Opus 4.5 (Thinking) Quote
26 | Claude Opus 4.5 (Thinking) | 85.32%±1.90 | $5/$25 | 72.11s
85.32% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 26 of 105 (Claude Opus 4.5 (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-5-20251101-thinking"]. Source of record MedScribe (Vals AI) MiMo V2.6 Flash Quote
27 | MiMo V2.6 Flash | 85.28%±1.99 | $0.14/$0.28 | 74.48s
85.28% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 27 of 105 (MiMo V2.6 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["xiaomi/mimo-v2.6-flash"]. Source of record MedScribe (Vals AI) Claude Haiku 4.5 (Thinking) Quote
29 | Claude Haiku 4.5 (Thinking) | 85.23%±1.90 | $1/$5 | 66.20s
85.23% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 29 of 105 (Claude Haiku 4.5 (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-haiku-4-5-20251001-thinking"]. Source of record MedScribe (Vals AI) GPT-5.6 Sol Quote
28 | GPT-5.6 Sol | 85.23%±1.97 | $4/$20 | 94.40s
85.23% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 28 of 105 (GPT-5.6 Sol), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.6-sol"]. Source of record MedScribe (Vals AI) Qwen 3.8 Max Quote
30 | Qwen 3.8 Max | 84.95%±2.00 | $2/$6 | 4m11s
84.95% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 30 of 105 (Qwen 3.8 Max), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["alibaba/qwen3.8-max"]. Source of record MedScribe (Vals AI) Claude Sonnet 4.5 (Nonthinking) Quote
31 | Claude Sonnet 4.5 (Nonthinking) | 84.52%±1.93 | $3/$15 | 44.42s
84.52% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 31 of 105 (Claude Sonnet 4.5 (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-5-20250929"]. Source of record MedScribe (Vals AI) Gemini 3.8 Flash Quote
32 | Gemini 3.8 Flash | 84.50%±1.94 | $1.5/$7.5 | 21.14s
84.50% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 32 of 105 (Gemini 3.8 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3.8-flash"]. Source of record MedScribe (Vals AI) GPT 5.2 Quote
34 | GPT 5.2 | 84.39%±1.86 | $1.75/$14 | 2m06s
84.39% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 34 of 105 (GPT 5.2), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.2-2025-12-11"]. Source of record MedScribe (Vals AI) GPT-5.6 Luna Quote
33 | GPT-5.6 Luna | 84.39%±2.58 | $0.2/$1.2 | 116.89s
84.39% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 33 of 105 (GPT-5.6 Luna), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.6-luna"]. Source of record MedScribe (Vals AI) Inkling Small Quote
35 | Inkling Small | 84.11%±1.87 | $0.3/$1.2 | 3m50s
84.11% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 35 of 105 (Inkling Small), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["thinkingmachines/inkling-small"]. Source of record MedScribe (Vals AI) Claude Sonnet 4.5 (Thinking) Quote
36 | Claude Sonnet 4.5 (Thinking) | 84.10%±1.87 | $3/$15 | 67.92s
84.10% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 36 of 105 (Claude Sonnet 4.5 (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-5-20250929-thinking"]. Source of record MedScribe (Vals AI) Gemini 3.7 Flash Quote
37 | Gemini 3.7 Flash | 83.94%±2.00 | $1.5/$7.5 | 16.82s
83.94% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 37 of 105 (Gemini 3.7 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3.7-flash"]. Source of record MedScribe (Vals AI) Qwen 3.8 27B Quote
38 | Qwen 3.8 27B | 83.85%±1.98 | $0.5/$3 | 89.70s
83.85% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 38 of 105 (Qwen 3.8 27B), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["alibaba/qwen3.8-27b"]. Source of record MedScribe (Vals AI) MiMo V2.5 Pro Quote
39 | MiMo V2.5 Pro | 83.73%±2.06 | $0.435/$0.87 | 90.71s
83.73% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 39 of 105 (MiMo V2.5 Pro), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["xiaomi/mimo-v2.5-pro"]. Source of record MedScribe (Vals AI) GPT-6 Luna Quote
40 | GPT-6 Luna | 83.71%±1.95 | $0.1/$0.5 | 2m50s
83.71% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 40 of 105 (GPT-6 Luna), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-6-luna"]. Source of record MedScribe (Vals AI) GPT 5 Quote
41 | GPT 5 | 83.65%±1.94 | $1.25/$10 | 3m05s
83.65% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 41 of 105 (GPT 5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5-2025-08-07"]. Source of record MedScribe (Vals AI) Hy4 Preview Quote
42 | Hy4 Preview | 83.60%±2.06 | $0.834/$2.501 | 5m50s
83.60% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 42 of 105 (Hy4 Preview), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["tencent/hy4-preview"]. Source of record MedScribe (Vals AI) GLM 5.2 Quote
43 | GLM 5.2 | 83.53%±2.00 | $1.4/$4.4 | 2m18s
83.53% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 43 of 105 (GLM 5.2), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["zai/glm-5.2"]. Source of record MedScribe (Vals AI) Claude Opus 4.5 (Nonthinking) Quote
44 | Claude Opus 4.5 (Nonthinking) | 83.25%±1.93 | $5/$25 | 43.30s
83.25% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 44 of 105 (Claude Opus 4.5 (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-5-20251101"]. Source of record MedScribe (Vals AI) Gemini 2.5 Flash (7/17) (Thinking) Quote
45 | Gemini 2.5 Flash (7/17) (Thinking) | 82.98%±1.91 | $0.3/$2.5 | 22.53s
82.98% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 45 of 105 (Gemini 2.5 Flash (7/17) (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-flash-thinking"]. Source of record MedScribe (Vals AI) Claude Opus 4.7 Quote
46 | Claude Opus 4.7 | 82.95%±1.98 | $5/$25 | 67.50s
82.95% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 46 of 105 (Claude Opus 4.7), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-7"]. Source of record MedScribe (Vals AI) Gemini 2.5 Flash (7/17) (Nonthinking) Quote
47 | Gemini 2.5 Flash (7/17) (Nonthinking) | 82.87%±1.91 | $0.3/$2.5 | 22.79s
82.87% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 47 of 105 (Gemini 2.5 Flash (7/17) (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-flash"]. Source of record MedScribe (Vals AI) GPT-5.6 Terra Quote
48 | GPT-5.6 Terra | 82.87%±1.95 | $2/$12 | 35.53s
82.87% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 48 of 105 (GPT-5.6 Terra), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.6-terra"]. Source of record MedScribe (Vals AI) GPT-6 Sol Quote
49 | GPT-6 Sol | 82.03%±1.94 | $2/$10 | 81.14s
82.03% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 49 of 105 (GPT-6 Sol), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-6-sol"]. Source of record MedScribe (Vals AI) Grok 4 Fast (Reasoning) Quote
50 | Grok 4 Fast (Reasoning) | 81.63%±2.14 | $0.2/$0.5 | 12.42s
81.63% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 50 of 105 (Grok 4 Fast (Reasoning)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4-fast-reasoning"]. Source of record MedScribe (Vals AI) Ling 3.0 Flash Quote
51 | Ling 3.0 Flash | 80.90%±2.04 | $0.075/$0.22 | 12.12s
80.90% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 51 of 105 (Ling 3.0 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["ant/ling-3.0-flash-2607"]. Source of record MedScribe (Vals AI) MiniMax-M2.1 Quote
52 | MiniMax-M2.1 | 80.78%±1.83 | $0.3/$1.2 | 53.16s
80.78% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 52 of 105 (MiniMax-M2.1), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["minimax/MiniMax-M2.1"]. Source of record MedScribe (Vals AI) GPT 5 Mini Quote
53 | GPT 5 Mini | 80.58%±1.92 | $0.25/$2 | 4m36s
80.58% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 53 of 105 (GPT 5 Mini), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5-mini-2025-08-07"]. Source of record MedScribe (Vals AI) DeepSeek V4 Flash 0731 Quote
54 | DeepSeek V4 Flash 0731 | 80.36%±1.97 | $0.44/$1.32 | 77.55s
80.36% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 54 of 105 (DeepSeek V4 Flash 0731), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["deepseek/deepseek-v4-flash-0731"]. Source of record MedScribe (Vals AI) DeepSeek V4 Pro 0813 Quote
55 | DeepSeek V4 Pro 0813 | 80.17%±2.00 | $1.32/$3.96 | 2m36s
80.17% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 55 of 105 (DeepSeek V4 Pro 0813), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["deepseek/deepseek-v4-pro-0813"]. Source of record MedScribe (Vals AI) MiniMax-M2.7 Quote
56 | MiniMax-M2.7 | 79.87%±1.86 | $0.3/$1.2 | 27.25s
79.87% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 56 of 105 (MiniMax-M2.7), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["minimax/MiniMax-M2.7"]. Source of record MedScribe (Vals AI) Grok 4 Fast (Non-Reasoning) Quote
57 | Grok 4 Fast (Non-Reasoning) | 79.72%±1.87 | $0.2/$0.5 | 8.48s
79.72% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 57 of 105 (Grok 4 Fast (Non-Reasoning)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4-fast-non-reasoning"]. Source of record MedScribe (Vals AI) Gemini 3.6 Flash Quote
58 | Gemini 3.6 Flash | 79.66%±1.86 | $1.5/$7.5 | 33.55s
79.66% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 58 of 105 (Gemini 3.6 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3.6-flash"]. Source of record MedScribe (Vals AI) Qwen 3.7 Max Quote
59 | Qwen 3.7 Max | 79.40%±1.91 | $2.5/$7.5 | 108.05s
79.40% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 59 of 105 (Qwen 3.7 Max), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["alibaba/qwen3.7-max"]. Source of record MedScribe (Vals AI) Grok 4.1 Fast (Reasoning) Quote
60 | Grok 4.1 Fast (Reasoning) | 78.73%±1.87 | $0.2/$0.5 | 39.29s
78.73% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 60 of 105 (Grok 4.1 Fast (Reasoning)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4-1-fast-reasoning"]. Source of record MedScribe (Vals AI) Gemini 2.5 Flash Preview (9/25) (Thinking) Quote
61 | Gemini 2.5 Flash Preview (9/25) (Thinking) | 78.50%±1.99 | $0.3/$2.5 | 31.39s
78.50% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 61 of 105 (Gemini 2.5 Flash Preview (9/25) (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-flash-preview-09-2025-thinking"]. Source of record MedScribe (Vals AI) Grok 4 Quote
62 | Grok 4 | 78.15%±2.08 | $3/$15 | 74.05s
78.15% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 62 of 105 (Grok 4), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4-0709"]. Source of record MedScribe (Vals AI) Kimi K2.6 Quote
63 | Kimi K2.6 | 78.15%±1.79 | $0.95/$4 | 8m14s
78.15% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 63 of 105 (Kimi K2.6), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["kimi/kimi-k2.6"]. Source of record MedScribe (Vals AI) Gemini 2.5 Flash Preview (9/25) (Nonthinking) Quote
64 | Gemini 2.5 Flash Preview (9/25) (Nonthinking) | 77.95%±1.92 | $0.3/$2.5 | 22.89s
77.95% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 64 of 105 (Gemini 2.5 Flash Preview (9/25) (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-flash-preview-09-2025"]. Source of record MedScribe (Vals AI) GPT 5.4 (xhigh) Quote
65 | GPT 5.4 (xhigh) | 77.55%±3.32 | $2.5/$15 | 4m43s
77.55% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 65 of 105 (GPT 5.4 (xhigh)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.4-2026-03-05"]. Source of record MedScribe (Vals AI) Grok 4.1 Fast Non-Reasoning Quote
66 | Grok 4.1 Fast Non-Reasoning | 77.46%±2.04 | $0.2/$0.5 | 19.86s
77.46% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 66 of 105 (Grok 4.1 Fast Non-Reasoning), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4-1-fast-non-reasoning"]. Source of record MedScribe (Vals AI) Qwen 3 VL Plus Quote
67 | Qwen 3 VL Plus | 77.13%±1.92 | $0.2/$1.6 | 71.35s
77.13% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 67 of 105 (Qwen 3 VL Plus), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["alibaba/qwen3-vl-plus-2025-09-23"]. Source of record MedScribe (Vals AI) GPT 5.4 Nano Quote
68 | GPT 5.4 Nano | 77.09%±1.89 | $0.2/$1.25 | 20.53s
77.09% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 68 of 105 (GPT 5.4 Nano), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5.4-nano-2026-03-17"]. Source of record MedScribe (Vals AI) Qwen 3.6 Plus Quote
69 | Qwen 3.6 Plus | 76.96%±1.92 | $0.5/$3 | 2m54s
76.96% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 69 of 105 (Qwen 3.6 Plus), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["alibaba/qwen3.6-plus"]. Source of record MedScribe (Vals AI) o3 Quote
70 | o3 | 76.65%±1.87 | $2/$8 | 46.39s
76.65% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 70 of 105 (o3), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/o3-2025-04-16"]. Source of record MedScribe (Vals AI) Gemini 3.5 Flash Quote
71 | Gemini 3.5 Flash | 76.57%±1.92 | $1.5/$9 | 57.08s
76.57% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 71 of 105 (Gemini 3.5 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3.5-flash"]. Source of record MedScribe (Vals AI) Kimi K2.5 Quote
72 | Kimi K2.5 | 76.44%±1.99 | $0.6/$3 | 2m27s
76.44% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 72 of 105 (Kimi K2.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["kimi/kimi-k2.5-thinking"]. Source of record MedScribe (Vals AI) Gemini 3.1 Pro Preview (02/26) Quote
73 | Gemini 3.1 Pro Preview (02/26) | 76.11%±1.92 | $2/$12 | 69.14s
76.11% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 73 of 105 (Gemini 3.1 Pro Preview (02/26)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3.1-pro-preview"]. Source of record MedScribe (Vals AI) Claude Sonnet 5 Quote
74 | Claude Sonnet 5 | 76.05%±3.05 | $2/$10 | 4m12s
76.05% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 74 of 105 (Claude Sonnet 5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-sonnet-5"]. Source of record MedScribe (Vals AI) Gemini 2.5 Flash Lite (9/25) (Nonthinking) Quote
75 | Gemini 2.5 Flash Lite (9/25) (Nonthinking) | 75.82%±1.85 | $0.1/$0.4 | 4.49s
75.82% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 75 of 105 (Gemini 2.5 Flash Lite (9/25) (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-flash-lite-preview-09-2025"]. Source of record MedScribe (Vals AI) Ling 3.0 Flash Fin Quote
76 | Ling 3.0 Flash Fin | 75.59%±2.03 | $0.06/$0.18 | 44.81s
75.59% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 76 of 105 (Ling 3.0 Flash Fin), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["ant/ling-3.0-flash-af-rc3"]. Source of record MedScribe (Vals AI) DeepSeek V4 Quote
77 | DeepSeek V4 | 75.14%±2.00 | $1.32/$3.96 | 5m46s
75.14% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 77 of 105 (DeepSeek V4), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["deepseek/deepseek-v4-pro"]. Source of record MedScribe (Vals AI) Grok 4.3 Quote
78 | Grok 4.3 | 74.40%±2.02 | $1.25/$2.5 | 100.46s
74.40% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 78 of 105 (Grok 4.3), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4.3"]. Source of record MedScribe (Vals AI) Claude Opus 4.1 (Thinking) Quote
79 | Claude Opus 4.1 (Thinking) | 73.90%±1.97 | $15/$75 | 57.40s
73.90% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 79 of 105 (Claude Opus 4.1 (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-1-20250805-thinking"]. Source of record MedScribe (Vals AI) Gemini 2.5 Pro Quote
80 | Gemini 2.5 Pro | 73.55%±1.91 | $1.25/$10 | 35.91s
73.55% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 80 of 105 (Gemini 2.5 Pro), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-pro"]. Source of record MedScribe (Vals AI) GPT 5 Nano Quote
81 | GPT 5 Nano | 72.86%±1.89 | $0.05/$0.4 | 112.91s
72.86% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 81 of 105 (GPT 5 Nano), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/gpt-5-nano-2025-08-07"]. Source of record MedScribe (Vals AI) Gemini 2.5 Flash Lite (Nonthinking) Quote
82 | Gemini 2.5 Flash Lite (Nonthinking) | 72.83%±1.98 | $0.1/$0.4 | 4.85s
72.83% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 82 of 105 (Gemini 2.5 Flash Lite (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-flash-lite"]. Source of record MedScribe (Vals AI) Qwen 3 Max Thinking Quote
83 | Qwen 3 Max Thinking | 72.71%±1.91 | $1.2/$6 | 6m02s
72.71% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 83 of 105 (Qwen 3 Max Thinking), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["alibaba/qwen3-max-2026-01-23"]. Source of record MedScribe (Vals AI) Claude Sonnet 4 (Nonthinking) Quote
84 | Claude Sonnet 4 (Nonthinking) | 72.41%±1.93 | $3/$15 | 25.67s
72.41% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 84 of 105 (Claude Sonnet 4 (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-20250514"]. Source of record MedScribe (Vals AI) GLM 5.1 Quote
85 | GLM 5.1 | 72.27%±2.06 | $1/$3.2 | 95.70s
72.27% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 85 of 105 (GLM 5.1), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["zai/glm-5.1"]. Source of record MedScribe (Vals AI) MiMo V2.5 Quote
86 | MiMo V2.5 | 72.15%±1.85 | $0.14/$0.28 | 20.26s
72.15% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 86 of 105 (MiMo V2.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["xiaomi/mimo-v2.5"]. Source of record MedScribe (Vals AI) Gemini 3 Pro (11/25) Quote
87 | Gemini 3 Pro (11/25) | 72.04%±1.90 | $2/$12 | 43.39s
72.04% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 87 of 105 (Gemini 3 Pro (11/25)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3-pro-preview"]. Source of record MedScribe (Vals AI) Claude Opus 4.1 (Nonthinking) Quote
88 | Claude Opus 4.1 (Nonthinking) | 71.75%±2.02 | $15/$75 | 38.04s
71.75% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 88 of 105 (Claude Opus 4.1 (Nonthinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-opus-4-1-20250805"]. Source of record MedScribe (Vals AI) Gemini 3.5 Flash Lite Quote
89 | Gemini 3.5 Flash Lite | 70.89%±2.03 | $0.3/$2.5 | 18.99s
70.89% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 89 of 105 (Gemini 3.5 Flash Lite), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3.5-flash-lite"]. Source of record MedScribe (Vals AI) Qwen 3.5 Flash Quote
90 | Qwen 3.5 Flash | 70.62%±2.09 | $0.1/$0.4 | 80.41s
70.62% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 90 of 105 (Qwen 3.5 Flash), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["alibaba/qwen3.5-flash"]. Source of record MedScribe (Vals AI) Gemini 3 Flash (12/25) Quote
91 | Gemini 3 Flash (12/25) | 69.92%±1.90 | $0.5/$3 | 23.49s
69.92% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 91 of 105 (Gemini 3 Flash (12/25)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3-flash-preview"]. Source of record MedScribe (Vals AI) Claude Sonnet 4 (Thinking) Quote
92 | Claude Sonnet 4 (Thinking) | 69.35%±2.21 | $3/$15 | 39.57s
69.35% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 92 of 105 (Claude Sonnet 4 (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["anthropic/claude-sonnet-4-20250514-thinking"]. Source of record MedScribe (Vals AI) o4 Mini Quote
93 | o4 Mini | 69.14%±1.96 | $1.1/$4.4 | 81.96s
69.14% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 93 of 105 (o4 Mini), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["openai/o4-mini-2025-04-16"]. Source of record MedScribe (Vals AI) GLM 4.7 Quote
94 | GLM 4.7 | 68.63%±2.12 | $0.6/$2.2 | 2m47s
68.63% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 94 of 105 (GLM 4.7), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["zai/glm-4.7"]. Source of record MedScribe (Vals AI) Mistral Medium 3.5 Quote
95 | Mistral Medium 3.5 | 67.73%±2.01 | $1.5/$7.5 | 104.05s
67.73% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 95 of 105 (Mistral Medium 3.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["mistralai/mistral-medium-3.5"]. Source of record MedScribe (Vals AI) Gemini 2.5 Flash Lite (9/25) (Thinking) Quote
96 | Gemini 2.5 Flash Lite (9/25) (Thinking) | 66.88%±1.92 | $0.1/$0.4 | 11.52s
66.88% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 96 of 105 (Gemini 2.5 Flash Lite (9/25) (Thinking)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-2.5-flash-lite-preview-09-2025-thinking"]. Source of record MedScribe (Vals AI) Laguna M.1 Quote
97 | Laguna M.1 | 65.91%±2.01 | N/A | 111.82s
65.91% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 97 of 105 (Laguna M.1), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["poolside/laguna-m.1"]. Source of record MedScribe (Vals AI) Gemini 3.1 Flash Lite Preview Quote
98 | Gemini 3.1 Flash Lite Preview | 63.90%±1.82 | $0.25/$1.5 | 16.79s
63.90% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 98 of 105 (Gemini 3.1 Flash Lite Preview), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["google/gemini-3.1-flash-lite-preview"]. Source of record MedScribe (Vals AI) Grok 4.20 (Reasoning) Quote
99 | Grok 4.20 (Reasoning) | 63.41%±2.10 | $2/$6 | 18.60s
63.41% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 99 of 105 (Grok 4.20 (Reasoning)), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["grok/grok-4.20-0309-reasoning"]. Source of record MedScribe (Vals AI) Laguna XS.2 Quote
100 | Laguna XS.2 | 61.43%±2.35 | N/A | 68.82s
61.43% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 100 of 105 (Laguna XS.2), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["poolside/laguna-xs.2"]. Source of record MedScribe (Vals AI) Command A+ Quote
101 | Command A+ | 55.68%±3.65 | N/A | 3m23s
55.68% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 101 of 105 (Command A+), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["cohere/command-a-plus-05-2026"]. Source of record MedScribe (Vals AI) Mercury 2.5 Quote
102 | Mercury 2.5 | 55.09%±2.10 | $0.2/$0.75 | 9.99s
55.09% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 102 of 105 (Mercury 2.5), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["inception/mercury-2.5"]. Source of record MedScribe (Vals AI) Llama 4 Maverick Quote
103 | Llama 4 Maverick | 54.22%±1.87 | $0.22/$0.88 | 25.05s
54.22% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 103 of 105 (Llama 4 Maverick), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["fireworks/llama4-maverick-instruct-basic"]. Source of record MedScribe (Vals AI) Llama 4 Scout Quote
104 | Llama 4 Scout | 50.59%±1.90 | $0.18/$0.59 | 11.32s
50.59% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 104 of 105 (Llama 4 Scout), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["together/meta-llama/Llama-4-Scout-17B-16E-Instruct"]. Source of record MedScribe (Vals AI) Nemotron 3.5 Lightning Quote
105 | Nemotron 3.5 Lightning | 4.27%±0.49 | $0.05/$0.2 | 105.52s
4.27% Vals AI MedScribe leaderboard, View: All Models, Task: Overall, row 105 of 105 (Nemotron 3.5 Lightning), Accuracy column; Updated 9/29/2026. Rendered BenchmarkView table; configuration from embedded astro-island BenchmarkView props, benchmarkView.default.tasks.overall["fireworks/nemotron-lightning-3p5-30b-a3b"]. Source of record MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results)ARISE AI Research Network, first-party, 15 Aug 202612 results
Open the document arise-ai.org, retrieved 7 Sep 2026
Board Row Printed Where Role First, Do NOHARM (v2) Muse Spark 1.1 Quote
5Muse Spark 1.1Meta 79.7%
79.7% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view) Source of record First, Do NOHARM (v2) Claude Opus 5 Quote
6Claude Opus 5Anthropic 74.6%
74.6% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column Source of record First, Do NOHARM (v2) Kimi K3 Quote
7Kimi K3OSSMoonshot AI 74.0%
74.0% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column Source of record First, Do NOHARM (v2) GPT-5.6 Sol Quote
8GPT-5.6 SolOpenAI 70.1%
70.1% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column Source of record First, Do NOHARM (v2) GPT-5.5 Quote
9GPT-5.5OpenAI 70.0%
70.0% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column Source of record First, Do NOHARM (v2) GPT-5 Quote
10 GPT-5 72.6%±2.3 68.6%±4.5 80.3%±4.4 30.1%±9.0 44.2%±2.5 45.4%±1.3 73.1%±3.9 73.7%±2.7 46.6%±0.0
68.6% arise-ai.org/mast/technical, Model Leaderboard (Top 10 shown), row 10, SAFETY column Source of record First, Do NOHARM (v2) Claude Fable 5 Quote
10Claude Fable 5Anthropic 65.0%
65.0% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view) Source of record First, Do NOHARM (v2) Gemini 3.1 Pro Quote
11Gemini 3.1 ProGoogle 62.6%
62.6% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column Source of record First, Do NOHARM (v2) Gemini 2.5 Pro Quote
12Gemini 2.5 ProGoogle 61.9%
61.9% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view) Source of record First, Do NOHARM (v2) Qwen3.5 397B A17B Quote
13Qwen3.5 397B A17BOSSAlibaba 61.1%
61.1% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view) Source of record First, Do NOHARM (v2) Kimi K2.6 Quote
14Kimi K2.6OSSMoonshot AI 59.1%
59.1% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view) Source of record First, Do NOHARM (v2) DeepSeek R1 Quote
16DeepSeek R1OSSDeepSeek 55.8%
55.8% arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view) Source of record MAST: Medical AI Superintelligence Test leaderboard (General board)ARISE AI Research Network, first-party, 15 Aug 20268 results
Open the document arise-ai.org, retrieved 7 Sep 2026
Board Row Printed Where Role MAST (Medical AI Superintelligence Test) GPT-5.6 Sol Quote
1 GPT-5.6 Sol OpenAI 60.2%
60.2% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record MAST (Medical AI Superintelligence Test) Kimi K3 Quote
2 Kimi K3 Moonshot AI 60.1%
60.1% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record MAST (Medical AI Superintelligence Test) Gemini 3.6 Flash Quote
3 Gemini 3.6 Flash Google 59.3%
59.3% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record MAST (Medical AI Superintelligence Test) Gemini 3.1 Pro Quote
4 Gemini 3.1 Pro Google 58.9%
58.9% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record MAST (Medical AI Superintelligence Test) Qwen3.5 397B A17B Quote
5 Qwen3.5 397B A17B Alibaba 57.9%
57.9% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record MAST (Medical AI Superintelligence Test) Claude Opus 5 Quote
6 Claude Opus 5 Anthropic 57.1%
57.1% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record MAST (Medical AI Superintelligence Test) Claude Sonnet 5 Quote
7 Claude Sonnet 5 Anthropic 56.6%
56.6% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record MAST (Medical AI Superintelligence Test) Grok 4.3 Quote
8 Grok 4.3 xAI 53.7%
53.7% arise-ai.org/mast, 'Which AI can you trust for medical questions?' General tab, composite score table (8 of 11 models shown; 'Last updated August 15, 2026') Source of record CHI-Bench leaderboard (actAVA)actAVA, first-party, 12 Aug 202643 results
Open the document actava.ai, retrieved 7 Sep 2026
Board Row Printed Where Role CHI-Bench erius + claude-opus-5 Quote
01 | erius submitted by Michael Johnson (MJ) | claude-opus-5 | Proprietary | 54.7% | 72.0% | 36.0% | 56.0% | 2026-07-26
54.7% CHI-Bench v1.0.0, All Domains, rank 01, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-26; accessed 2026-09-30. Source of record CHI-Bench claude-code + claude-opus-5 Quote
03 | claude-code | claude-opus-5 | Proprietary | 37.3% | 20.0% | 32.0% | 60.0% | 2026-07-24
37.3% CHI-Bench v1.0.0, All Domains, rank 03, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-24T19:08:02Z to 2026-07-24T23:22:36Z; submitted_at 2026-08-12T20:20:00Z. Source of record CHI-Bench erius + claude-opus-4-8 Quote
02 | erius submitted by Michael Johnson (MJ) | claude-opus-4-8 | Proprietary | 37.3% | 40.0% | 16.0% | 56.0% | 2026-06-05
37.3% CHI-Bench v1.0.0, All Domains, rank 02, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-06-05; accessed 2026-09-30. Source of record CHI-Bench claude-code + claude-opus-4-8 Quote
04 | claude-code | claude-opus-4-8 | Proprietary | 33.3% | 32.0% | 28.0% | 40.0% | 2026-05-28
33.3% CHI-Bench v1.0.0, All Domains, rank 04, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-28; accessed 2026-09-30. Linked submission provenance: run 2026-05-28T17:19:15Z to 2026-05-28T20:48:59Z; submitted_at 2026-05-28T20:53:26Z. Source of record CHI-Bench claude-code + claude-opus-4-6 Quote
05 | claude-code | claude-opus-4-6 | Proprietary | 28.0% | 20.0% | 36.0% | 28.0% | 2026-05-01
28.0% CHI-Bench v1.0.0, All Domains, rank 05, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-12T18:55:45Z to 2026-05-12T18:55:46Z; submitted_at 2026-05-12T22:39:33Z. Source of record CHI-Bench claude-code + claude-sonnet-4-6 Quote
06 | claude-code | claude-sonnet-4-6 | Proprietary | 26.2% | 24.0% | 34.7% | 20.0% | 2026-05-01
26.2% CHI-Bench v1.0.0, All Domains, rank 06, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T14:43:27Z to 2026-04-30T15:00:19Z; submitted_at 2026-04-30T15:00:19Z. Source of record CHI-Bench codex + gpt-5.6-sol Quote
07 | codex | gpt-5.6-sol | Proprietary | 25.3% | 36.0% | 28.0% | 12.0% | 2026-07-24
25.3% CHI-Bench v1.0.0, All Domains, rank 07, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T02:41:32.718905Z to 2026-07-22T04:52:39.419746Z; submitted_at 2026-07-24T23:01:25Z. Source of record CHI-Bench openai-agents + kimi-k3 Quote
08 | openai-agents | kimi-k3 | Open-source | 25.3% | 28.0% | 32.0% | 16.0% | 2026-07-24
25.3% CHI-Bench v1.0.0, All Domains, rank 08, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T08:03:23.788878Z to 2026-07-22T14:22:38.269923Z; submitted_at 2026-08-12T20:20:00Z. Source of record CHI-Bench claude-code + claude-opus-4-7 Quote
09 | claude-code | claude-opus-4-7 | Proprietary | 24.4% | 24.0% | 17.3% | 32.0% | 2026-05-01
24.4% CHI-Bench v1.0.0, All Domains, rank 09, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T06:06:32Z to 2026-04-30T10:00:03Z; submitted_at 2026-04-30T10:00:03Z. Source of record CHI-Bench claude-code + claude-fable-5 Quote
10 | claude-code | claude-fable-5 | Proprietary | 24.0% | 24.0% | 24.0% | 24.0% | 2026-07-22
24.0% CHI-Bench v1.0.0, All Domains, rank 10, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-22; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T07:16:27.207645Z to 2026-07-22T10:25:29.658582Z; submitted_at 2026-07-22T23:50:25Z. Source of record CHI-Bench hermes + MedGuard Quote
11 | hermes submitted by cuilinke | MedGuard | Open-source | 22.7% | 4.0% | 4.0% | 60.0% | 2026-07-06
22.7% CHI-Bench v1.0.0, All Domains, rank 11, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-01T06:59:37Z to 2026-07-01T08:17:44Z; submitted_at 2026-07-06T09:10:18Z. Source of record CHI-Bench codex + gpt-5.5 Quote
12 | codex | gpt-5.5 | Proprietary | 20.9% | 29.3% | 32.0% | 1.3% | 2026-05-01
20.9% CHI-Bench v1.0.0, All Domains, rank 12, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T06:06:32Z to 2026-04-30T09:41:04Z; submitted_at 2026-04-30T09:41:04Z. Source of record CHI-Bench claude-code + claude-sonnet-5 Quote
13 | claude-code | claude-sonnet-5 | Proprietary | 20.0% | 24.0% | 24.0% | 12.0% | 2026-07-06
20.0% CHI-Bench v1.0.0, All Domains, rank 13, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-02T04:22:04Z to 2026-07-02T04:31:46Z; submitted_at 2026-07-06T05:23:12Z. Source of record CHI-Bench hermes + glm-5.1 Quote
15 | hermes | glm-5.1 | Open-source | 18.7% | 10.7% | 34.7% | 10.7% | 2026-05-01
18.7% CHI-Bench v1.0.0, All Domains, rank 15, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:01:48Z to 2026-05-04T05:47:30Z; submitted_at 2026-05-04T05:47:30Z. Source of record CHI-Bench openai-agents + glm-5.1 Quote
14 | openai-agents | glm-5.1 | Open-source | 18.7% | 18.7% | 33.3% | 4.0% | 2026-05-01
18.7% CHI-Bench v1.0.0, All Domains, rank 14, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T08:41:54Z to 2026-05-04T07:03:18Z; submitted_at 2026-05-04T07:03:18Z. Source of record CHI-Bench openai-agents + glm-5.2 Quote
16 | openai-agents | glm-5.2 | Open-source | 18.7% | 20.0% | 32.0% | 4.0% | 2026-07-06
18.7% CHI-Bench v1.0.0, All Domains, rank 16, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-06; accessed 2026-09-30. Linked submission provenance: run 2026-07-02T04:22:04Z to 2026-07-02T04:48:05Z; submitted_at 2026-07-06T05:23:05Z. Source of record CHI-Bench openclaw + claude-opus-4-7 Quote
17 | openclaw | claude-opus-4-7 | Proprietary | 17.3% | 18.7% | 13.3% | 20.0% | 2026-05-01
17.3% CHI-Bench v1.0.0, All Domains, rank 17, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-04T13:50:05Z to 2026-05-04T15:47:22Z; submitted_at 2026-05-04T15:47:22Z. Source of record CHI-Bench openclaw + glm-5.1 Quote
18 | openclaw | glm-5.1 | Open-source | 16.9% | 13.3% | 26.7% | 10.7% | 2026-05-01
16.9% CHI-Bench v1.0.0, All Domains, rank 18, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:10:57Z to 2026-05-04T02:49:29Z; submitted_at 2026-05-04T02:49:29Z. Source of record CHI-Bench hermes + qwen-3.6-max Quote
19 | hermes | qwen-3.6-max | Open-source | 16.4% | 9.3% | 26.7% | 13.3% | 2026-05-01
16.4% CHI-Bench v1.0.0, All Domains, rank 19, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:32:36Z to 2026-05-04T06:18:19Z; submitted_at 2026-05-04T06:18:19Z. Source of record CHI-Bench codex + gpt-5.4 Quote
20 | codex | gpt-5.4 | Proprietary | 16.0% | 24.0% | 17.3% | 6.7% | 2026-05-01
16.0% CHI-Bench v1.0.0, All Domains, rank 20, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-04-30T22:58:31Z; submitted_at 2026-04-30T22:58:31Z. Source of record CHI-Bench hermes + kimi-k2.6 Quote
22 | hermes | kimi-k2.6 | Open-source | 15.6% | 18.7% | 21.3% | 6.7% | 2026-05-01
15.6% CHI-Bench v1.0.0, All Domains, rank 22, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:19:12Z to 2026-05-04T06:35:44Z; submitted_at 2026-05-04T06:35:44Z. Source of record CHI-Bench openai-agents + qwen-3.6-max Quote
21 | openai-agents | qwen-3.6-max | Open-source | 15.6% | 16.0% | 26.7% | 4.0% | 2026-05-01
15.6% CHI-Bench v1.0.0, All Domains, rank 21, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:19:32Z to 2026-05-04T07:25:50Z; submitted_at 2026-05-04T07:25:50Z. Source of record CHI-Bench openai-agents + kimi-k2.6 Quote
23 | openai-agents | kimi-k2.6 | Open-source | 15.1% | 17.3% | 25.3% | 2.7% | 2026-05-01
15.1% CHI-Bench v1.0.0, All Domains, rank 23, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:00:39Z to 2026-05-04T07:47:06Z; submitted_at 2026-05-04T07:47:06Z. Source of record CHI-Bench openai-agents + deepseek-v4-pro Quote
24 | openai-agents | deepseek-v4-pro | Open-source | 14.2% | 10.7% | 28.0% | 4.0% | 2026-05-01
14.2% CHI-Bench v1.0.0, All Domains, rank 24, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T08:35:38Z to 2026-05-04T06:59:15Z; submitted_at 2026-05-04T06:59:15Z. Source of record CHI-Bench hermes + deepseek-v4-pro Quote
25 | hermes | deepseek-v4-pro | Open-source | 13.8% | 8.0% | 25.3% | 8.0% | 2026-05-01
13.8% CHI-Bench v1.0.0, All Domains, rank 25, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T06:28:45Z to 2026-05-04T05:51:44Z; submitted_at 2026-05-04T05:51:44Z. Source of record CHI-Bench codex + gpt-5.6-luna Quote
27 | codex | gpt-5.6-luna | Proprietary | 13.3% | 20.0% | 16.0% | 4.0% | 2026-07-24
13.3% CHI-Bench v1.0.0, All Domains, rank 27, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T06:23:21.720425Z to 2026-07-22T08:02:34.258993Z; submitted_at 2026-07-24T23:01:25Z. Source of record CHI-Bench codex + gpt-5.6-terra Quote
26 | codex | gpt-5.6-terra | Proprietary | 13.3% | 12.0% | 20.0% | 8.0% | 2026-07-24
13.3% CHI-Bench v1.0.0, All Domains, rank 26, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T04:52:42.763467Z to 2026-07-22T06:23:17.742720Z; submitted_at 2026-07-24T23:01:25Z. Source of record CHI-Bench gemini-cli + gemini-3-flash Quote
28 | gemini-cli | gemini-3-flash | Proprietary | 12.5% | 18.7% | 18.7% | 0.0% | 2026-05-01
12.5% CHI-Bench v1.0.0, All Domains, rank 28, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-30T09:24:38Z to 2026-04-30T20:11:40Z; submitted_at 2026-04-30T20:11:40Z. Source of record CHI-Bench deepagents + glm-5.1 Quote
30 | deepagents | glm-5.1 | Open-source | 11.1% | 17.3% | 10.7% | 5.3% | 2026-05-01
11.1% CHI-Bench v1.0.0, All Domains, rank 30, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T21:51:53Z to 2026-05-04T13:10:34Z; submitted_at 2026-05-04T13:10:34Z. Source of record CHI-Bench openclaw + deepseek-v4-pro Quote
29 | openclaw | deepseek-v4-pro | Open-source | 11.1% | 14.7% | 12.0% | 6.7% | 2026-05-01
11.1% CHI-Bench v1.0.0, All Domains, rank 29, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:01:51Z to 2026-05-04T03:00:01Z; submitted_at 2026-05-04T03:00:01Z. Source of record CHI-Bench deepagents + deepseek-v4-pro Quote
31 | deepagents | deepseek-v4-pro | Open-source | 10.7% | 14.7% | 10.7% | 6.7% | 2026-05-01
10.7% CHI-Bench v1.0.0, All Domains, rank 31, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:35:56Z to 2026-05-05T01:34:20Z; submitted_at 2026-05-05T01:34:20Z. Source of record CHI-Bench openclaw + kimi-k2.6 Quote
32 | openclaw | kimi-k2.6 | Open-source | 10.2% | 12.0% | 18.7% | 0.0% | 2026-05-01
10.2% CHI-Bench v1.0.0, All Domains, rank 32, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T06:28:45Z to 2026-05-04T05:50:33Z; submitted_at 2026-05-04T05:50:33Z. Source of record CHI-Bench deepagents + qwen-3.6-max Quote
33 | deepagents | qwen-3.6-max | Open-source | 9.3% | 12.0% | 10.7% | 5.3% | 2026-05-01
9.3% CHI-Bench v1.0.0, All Domains, rank 33, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T22:18:28Z to 2026-05-04T13:19:53Z; submitted_at 2026-05-04T13:19:53Z. Source of record CHI-Bench codex + gpt-5.4-mini Quote
34 | codex | gpt-5.4-mini | Proprietary | 8.4% | 10.7% | 13.3% | 1.3% | 2026-05-01
8.4% CHI-Bench v1.0.0, All Domains, rank 34, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-05-01T03:47:25Z; submitted_at 2026-05-01T03:47:25Z. Source of record CHI-Bench openai-agents + TML Inkling 256K Quote
35 | openai-agents | TML Inkling 256K | Open-source | 8.0% | 4.0% | 16.0% | 4.0% | 2026-07-24
8.0% CHI-Bench v1.0.0, All Domains, rank 35, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T14:25:50.976659Z to 2026-07-22T16:53:02.466832Z; submitted_at 2026-07-24T23:02:07Z. Source of record CHI-Bench claude-code + claude-haiku-4-5 Quote
37 | claude-code | claude-haiku-4-5 | Proprietary | 6.2% | 0.0% | 14.7% | 4.0% | 2026-05-01
6.2% CHI-Bench v1.0.0, All Domains, rank 37, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-04-29T19:36:29Z to 2026-04-30T19:34:41Z; submitted_at 2026-04-30T19:34:41Z. Source of record CHI-Bench openai-agents + grok-4.3 Quote
38 | openai-agents | grok-4.3 | Open-source | 5.8% | 0.0% | 16.0% | 1.3% | 2026-05-01
5.8% CHI-Bench v1.0.0, All Domains, rank 38, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T09:26:38Z to 2026-05-04T07:18:48Z; submitted_at 2026-05-04T07:18:48Z. Source of record CHI-Bench openclaw + qwen-3.6-max Quote
39 | openclaw | qwen-3.6-max | Open-source | 4.9% | 10.7% | 4.0% | 0.0% | 2026-05-01
4.9% CHI-Bench v1.0.0, All Domains, rank 39, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:26:43Z to 2026-05-04T03:15:50Z; submitted_at 2026-05-04T03:15:50Z. Source of record CHI-Bench hermes + grok-4.3 Quote
40 | hermes | grok-4.3 | Open-source | 4.4% | 0.0% | 13.3% | 0.0% | 2026-05-01
4.4% CHI-Bench v1.0.0, All Domains, rank 40, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T07:34:26Z to 2026-05-04T06:20:37Z; submitted_at 2026-05-04T06:20:37Z. Source of record CHI-Bench deepagents + kimi-k2.6 Quote
41 | deepagents | kimi-k2.6 | Open-source | 3.1% | 8.0% | 1.3% | 0.0% | 2026-05-01
3.1% CHI-Bench v1.0.0, All Domains, rank 41, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T21:58:07Z to 2026-05-04T13:39:21Z; submitted_at 2026-05-04T13:39:21Z. Source of record CHI-Bench deepagents + grok-4.3 Quote
42 | deepagents | grok-4.3 | Open-source | 2.2% | 0.0% | 5.3% | 1.3% | 2026-05-01
2.2% CHI-Bench v1.0.0, All Domains, rank 42, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-03T22:19:46Z to 2026-05-04T13:19:20Z; submitted_at 2026-05-04T13:19:20Z. Source of record CHI-Bench openclaw + grok-4.3 Quote
43 | openclaw | grok-4.3 | Open-source | 0.4% | 1.3% | 0.0% | 0.0% | 2026-05-01
0.4% CHI-Bench v1.0.0, All Domains, rank 43, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-05-01; accessed 2026-09-30. Linked submission provenance: run 2026-05-02T09:34:46Z to 2026-05-04T00:03:58Z; submitted_at 2026-05-04T00:03:58Z. Source of record CHI-Bench openai-agents + Nemotron 3 Ultra 256K Quote
44 | openai-agents | Nemotron 3 Ultra 256K | Open-source | 0.0% | 0.0% | 0.0% | 0.0% | 2026-07-24
0.0% CHI-Bench v1.0.0, All Domains, rank 44, Agent/Model and Accuracy columns; PA/UM/CM follow; board last updated 2026-08-12; board Date column 2026-07-24; accessed 2026-09-30. Linked submission provenance: run 2026-07-22T22:31:29.302007Z to 2026-07-22T23:37:41.111758Z; submitted_at 2026-08-12T20:20:00Z. Source of record HealthAgentBench detailed resultsMicrosoft Research (HealthAgentBench), first-party, 27 Jul 202611 results
Open the document microsoft.github.io, retrieved 7 Sep 2026
Board Row Printed Where Role HealthAgentBench Claude Code (Opus 5) Quote
1 Claude Code (Opus 5) 55%
55% Detailed results table, rank 1 Also prints it HealthAgentBench Codex (GPT-5.6-sol) Quote
2 Codex (GPT-5.6-sol) 45%
45% Detailed results table, rank 2 Also prints it HealthAgentBench Codex (GPT 5.5) Quote
3 Codex (GPT 5.5) 42%
42% Detailed results table, rank 3 Also prints it HealthAgentBench Copilot (Opus 4.8) Quote
4 Copilot (Opus 4.8) 36%
36% Detailed results table, rank 4 Also prints it HealthAgentBench Copilot (GPT 5.5) Quote
5 Copilot (GPT 5.5) 35%
35% Detailed results table, rank 5 Also prints it HealthAgentBench Claude Code (Opus 4.8) Quote
6 Claude Code (Opus 4.8) 32%
32% Detailed results table, rank 6 Also prints it HealthAgentBench Codex (GPT 5.4) Quote
7 Codex (GPT 5.4) 28%
28% Detailed results table, rank 7 Also prints it HealthAgentBench Claude Code (Opus 4.7) Quote
8 Claude Code (Opus 4.7) 27%
27% Detailed results table, rank 8 Also prints it HealthAgentBench Claude Code (Opus 4.6) Quote
10 Claude Code (Opus 4.6) 19%
19% Detailed results table, rank 10 Also prints it HealthAgentBench Claude Code (Sonnet 4.6) Quote
11 Claude Code (Sonnet 4.6) 17%
17% Detailed results table, rank 11 Also prints it HealthAgentBench Codex (GPT 5.4 Mini) Quote
12 Codex (GPT 5.4 Mini) 16%
16% Detailed results table, rank 12 Also prints it HealthAgentBench leaderboardMicrosoft Research (HealthAgentBench), first-party, 27 Jul 202612 results
Open the document microsoft.github.io, retrieved 7 Sep 2026
Board Row Printed Where Role HealthAgentBench Claude Code (Opus 5) Quote
1 Claude Code (Opus 5) 55% $3.3
55% Leaderboard table (homepage), rank 1, Success Rate column Source of record HealthAgentBench Codex (GPT-5.6-sol) Quote
2 Codex (GPT-5.6-sol) 45% $5.2
45% Leaderboard table (homepage), rank 2, Success Rate column Source of record HealthAgentBench Codex (GPT 5.5) Quote
3 Codex (GPT 5.5) 42% $2.8
42% Leaderboard table (homepage), rank 3, Success Rate column Source of record HealthAgentBench Copilot (Opus 4.8) Quote
4 Copilot (Opus 4.8) 36% $3.1
36% Leaderboard table (homepage), rank 4, Success Rate column Source of record HealthAgentBench Copilot (GPT 5.5) Quote
5 Copilot (GPT 5.5) 35% $2.6
35% Leaderboard table (homepage), rank 5, Success Rate column Source of record HealthAgentBench Claude Code (Opus 4.8) Quote
6 Claude Code (Opus 4.8) 32% $4.0
32% Leaderboard table (homepage), rank 6, Success Rate column Source of record HealthAgentBench Codex (GPT 5.4) Quote
7 Codex (GPT 5.4) 28% $1.3
28% Leaderboard table (homepage), rank 7, Success Rate column Source of record HealthAgentBench Claude Code (Opus 4.7) Quote
8 Claude Code (Opus 4.7) 27% $4.8
27% Leaderboard table (homepage), rank 8, Success Rate column Source of record HealthAgentBench Codex (GPT 5.3) Quote
Agent | Success Rate | cost/task Codex (GPT 5.3) 22% $1.0
22% Homepage leaderboard, rank 9, Success Rate and cost/task; displayed 2026-09-30. Source of record HealthAgentBench Claude Code (Opus 4.6) Quote
10 Claude Code (Opus 4.6) 19% $4.1
19% Leaderboard table (homepage), rank 10, Success Rate column Source of record HealthAgentBench Claude Code (Sonnet 4.6) Quote
11 Claude Code (Sonnet 4.6) 17% $2.9
17% Leaderboard table (homepage), rank 11, Success Rate column Source of record HealthAgentBench Codex (GPT 5.4 Mini) Quote
12 Codex (GPT 5.4 Mini) 16% $0.6
16% Leaderboard table (homepage), rank 12, Success Rate column Source of record MedHELM leaderboard (medhelm.org), v5.0.0Stanford CRFM (MedHELM), first-party, 14 May 202610 results
Open the document medhelm.org, retrieved 7 Sep 2026
Board Row Printed Where Role MedHELM Gemini 3.1 Pro (Preview) Quote
1 Gemini 3.1 Pro (Preview) Google 0.652
0.652 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM Gemini 3.5 Flash Quote
2 Gemini-3.5-flash Google 0.642
0.642 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM Muse Spark (2026-04-08) Quote
3 Muse Spark (2026-04-08) Meta 0.621
0.621 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM GPT-5.4 mini Quote
4 GPT-5.4 mini OpenAI 0.552
0.552 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM GPT-5.4 (2026-03-05) Quote
5 GPT-5.4 (2026-03-05) OpenAI 0.538
0.538 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM Gemini 2.5 Pro Quote
6 Gemini 2.5 Pro (05-06 preview) Google 0.529
0.529 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM DeepSeek R1 Quote
7 DeepSeek R1 DeepSeek 0.485
0.485 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM Claude 4.6 Opus Quote
8 Claude 4.6 Opus Anthropic 0.456
0.456 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM Claude 3.7 Sonnet Quote
9 Claude 3.7 Sonnet (20250219) Anthropic 0.45
0.45 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM Gemini 2.0 Flash Quote
10 Gemini 2.0 Flash Google 0.342
0.342 medhelm.org home, 'Current leaders Mean win rate v5.0.0' table ('10 of 11 models · Updated 14 May 2026') Source of record MedHELM v5.0.0 release data: medhelm_scenarios group table (mean win rate)Stanford CRFM (MedHELM), first-party, 8 May 202610 results
Open the document leaderboard.medhelm.org, retrieved 7 Sep 2026
Board Row Printed Where Role MedHELM Gemini 3.1 Pro (Preview) Quote
["Gemini 3.1 Pro (Preview)", "Mean win rate": 0.6520833333333333]
0.6520833333333333 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM Gemini 3.5 Flash Quote
["Gemini-3.5-flash", "Mean win rate": 0.6416666666666667]
0.6416666666666667 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM Muse Spark (2026-04-08) Quote
["Muse Spark (2026-04-08)", "Mean win rate": 0.6208333333333333]
0.6208333333333333 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM GPT-5.4 mini Quote
["GPT-5.4 mini", "Mean win rate": 0.5520833333333334]
0.5520833333333334 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM GPT-5.4 (2026-03-05) Quote
["GPT-5.4 (2026-03-05)", "Mean win rate": 0.5375]
0.5375 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM Gemini 2.5 Pro Quote
["Gemini 2.5 Pro (05-06 preview)", "Mean win rate": 0.5291666666666667]
0.5291666666666667 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM DeepSeek R1 Quote
["DeepSeek R1", "Mean win rate": 0.48541666666666666]
0.48541666666666666 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM Claude 4.6 Opus Quote
["Claude 4.6 Opus", "Mean win rate": 0.45625]
0.45625 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM Claude 3.7 Sonnet Quote
["Claude 3.7 Sonnet (20250219)", "Mean win rate": 0.45]
0.45 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it MedHELM Gemini 2.0 Flash Quote
["Gemini 2.0 Flash", "Mean win rate": 0.3416666666666667]
0.3416666666666667 releases/v5.0.0/groups/medhelm_scenarios.json, table 'Accuracy', column 'Mean win rate' Also prints it ARISE MAST technical leaderboardARISE, first-party, date not stated5 results
Open the document www.arise-ai.org
Board Row Printed Where Role First, Do NOHARM (v2) LiSA 2.5 Quote
First Do NOHARM v2 overall metric across 19 models 1LiSA 2.5RAGAMBOSS 86.2%
86.2% First Do NOHARM v2, overall leaderboard; LiSA 2.5 row; displayed 2026-09-30. Source of record First, Do NOHARM (v2) Doximity Ask 6.1 Quote
First Do NOHARM v2 overall metric across 19 models 2Doximity Ask 6.1RAGDoximity 84.5%
84.5% First Do NOHARM v2, overall leaderboard; Doximity Ask 6.1 row; displayed 2026-09-30. Source of record First, Do NOHARM (v2) OpenEvidence Quote
First Do NOHARM v2 overall metric across 19 models 3OpenEvidenceRAGOpenEvidence 80.0%
80.0% First Do NOHARM v2, overall leaderboard; OpenEvidence row; displayed 2026-09-30. Source of record First, Do NOHARM (v2) Glass 5.6 Max Quote
First Do NOHARM v2 overall metric across 19 models 4Glass 5.6 MaxRAGGlass Health 79.7%
79.7% First Do NOHARM v2, overall leaderboard; Glass 5.6 Max row; displayed 2026-09-30. Source of record First, Do NOHARM (v2) GLM 5.1 Quote
First Do NOHARM v2 overall metric across 19 models 15GLM 5.1OSSZ.ai 57.9%
57.9% First Do NOHARM v2, overall leaderboard; GLM 5.1 row; displayed 2026-09-30. Source of record CHI-Bench submission manifest: claude-code + anthropic/claude-fable-5actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-fable-5 Quote
anthropic/claude-fable-5
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: claude-code + anthropic/claude-haiku-4-5actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-haiku-4-5 Quote
anthropic/claude-haiku-4-5
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: claude-code + anthropic/claude-opus-4-6actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-opus-4-6 Quote
anthropic/claude-opus-4-6
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: claude-code + anthropic/claude-opus-4-7actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-opus-4-7 Quote
anthropic/claude-opus-4-7
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: claude-code + anthropic/claude-opus-4-8actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-opus-4-8 Quote
anthropic/claude-opus-4-8
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: claude-code + anthropic/claude-opus-5actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-opus-5 Quote
anthropic/claude-opus-5
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: claude-code + anthropic/claude-sonnet-4-6actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-sonnet-4-6 Quote
anthropic/claude-sonnet-4-6
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: claude-code + anthropic/claude-sonnet-5actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench claude-code + claude-sonnet-5 Quote
anthropic/claude-sonnet-5
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: codex + openai/gpt-5.4actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench codex + gpt-5.4 Quote
openai/gpt-5.4
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: codex + openai/gpt-5.4-miniactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench codex + gpt-5.4-mini Quote
openai/gpt-5.4-mini
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: codex + openai/gpt-5.5actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench codex + gpt-5.5 Quote
openai/gpt-5.5
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: codex + openai/gpt-5.6-lunaactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench codex + gpt-5.6-luna Quote
openai/gpt-5.6-luna
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: codex + openai/gpt-5.6-solactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench codex + gpt-5.6-sol Quote
openai/gpt-5.6-sol
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: codex + openai/gpt-5.6-terraactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench codex + gpt-5.6-terra Quote
openai/gpt-5.6-terra
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: deepagents + openrouter/deepseek/deepseek-v4-proactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench deepagents + deepseek-v4-pro Quote
openrouter/deepseek/deepseek-v4-pro
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: deepagents + openrouter/moonshotai/kimi-k2.6actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench deepagents + kimi-k2.6 Quote
openrouter/moonshotai/kimi-k2.6
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: deepagents + openrouter/qwen/qwen3-6-max-previewactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench deepagents + qwen-3.6-max Quote
openrouter/qwen/qwen3-6-max-preview
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: deepagents + openrouter/x-ai/grok-4.3actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench deepagents + grok-4.3 Quote
openrouter/x-ai/grok-4.3
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: deepagents + openrouter/z-ai/glm-5.1actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench deepagents + glm-5.1 Quote
openrouter/z-ai/glm-5.1
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: erius + anthropic/claude-opus-4-8actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench erius + claude-opus-4-8 Quote
anthropic/claude-opus-4-8
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: erius + anthropic/claude-opus-5actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench erius + claude-opus-5 Quote
anthropic/claude-opus-5
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: gemini-cli + google/gemini-3-flash-previewactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench gemini-cli + gemini-3-flash Quote
0.12444444444444443
– results.overall.pass_at_1 for submission.model google/gemini-3-flash-preview; fraction scale (not percent); retrieved 2026-09-30 Prints a different number CHI-Bench submission manifest: hermes + MedGuardactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench hermes + MedGuard Quote
MedGuard
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: hermes + openrouter/deepseek/deepseek-v4-proactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench hermes + deepseek-v4-pro Quote
openrouter/deepseek/deepseek-v4-pro
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: hermes + openrouter/moonshotai/kimi-k2.6actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench hermes + kimi-k2.6 Quote
openrouter/moonshotai/kimi-k2.6
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: hermes + openrouter/qwen/qwen3-6-max-previewactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench hermes + qwen-3.6-max Quote
openrouter/qwen/qwen3-6-max-preview
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: hermes + openrouter/x-ai/grok-4.3actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench hermes + grok-4.3 Quote
openrouter/x-ai/grok-4.3
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: hermes + openrouter/z-ai/glm-5.1actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench hermes + glm-5.1 Quote
openrouter/z-ai/glm-5.1
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + deepseek/deepseek-v4-proactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + deepseek-v4-pro Quote
deepseek/deepseek-v4-pro
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + moonshotai/kimi-k2.6actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + kimi-k2.6 Quote
moonshotai/kimi-k2.6
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + moonshotai/kimi-k3actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + kimi-k3 Quote
moonshotai/kimi-k3
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16:peft:262144actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + Nemotron 3 Ultra 256K Quote
nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16:peft:262144
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + qwen/qwen3-6-max-previewactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + qwen-3.6-max Quote
qwen/qwen3-6-max-preview
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + thinkingmachines/Inkling:peft:262144actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + TML Inkling 256K Quote
thinkingmachines/Inkling:peft:262144
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + x-ai/grok-4.3actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + grok-4.3 Quote
x-ai/grok-4.3
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + z-ai/glm-5.1actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + glm-5.1 Quote
z-ai/glm-5.1
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openai-agents + z-ai/glm-5.2actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openai-agents + glm-5.2 Quote
z-ai/glm-5.2
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openclaw + anthropic/claude-opus-4-7actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openclaw + claude-opus-4-7 Quote
anthropic/claude-opus-4-7
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openclaw + openrouter/deepseek/deepseek-v4-proactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openclaw + deepseek-v4-pro Quote
openrouter/deepseek/deepseek-v4-pro
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openclaw + openrouter/moonshotai/kimi-k2.6actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openclaw + kimi-k2.6 Quote
openrouter/moonshotai/kimi-k2.6
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openclaw + openrouter/qwen/qwen3-6-max-previewactAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openclaw + qwen-3.6-max Quote
openrouter/qwen/qwen3-6-max-preview
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openclaw + openrouter/x-ai/grok-4.3actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openclaw + grok-4.3 Quote
openrouter/x-ai/grok-4.3
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it CHI-Bench submission manifest: openclaw + openrouter/z-ai/glm-5.1actAVA / CHI-Bench submission repository, first-party, date not stated1 result
Open the document raw.githubusercontent.com, retrieved 30 Sep 2026
Board Row Printed Where Role CHI-Bench openclaw + glm-5.1 Quote
openrouter/z-ai/glm-5.1
– submission.model; provenance.started_at / finished_at; results.overall.pass_at_1; retrieved 2026-09-30 Also prints it
Paper
Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety ReasoningZhitian Hou, Yuhang Liu, Pengkai Wang, Zeyu Liu, Guanghao Zhu, Zheng Liu, Shuo Cai, Congkai Xie, Zhijie Sang, Kun Zeng, Hongxia Yang (The Hong Kong Polytechnic University; InfiX.ai; Sun Yat-sen University), third-party, 4 Aug 2026196 results
Open the document arxiv.org
Board Row Printed Where Role MedPIC Gemini-3.1-Pro Quote
Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
80.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC GPT-5 Quote
GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
75.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Qwen3.5-Plus Quote
Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
72.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC DeepSeek-V4-Pro Quote
DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
71.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC GPT-OSS-120B Quote
GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
70.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Qwen3.5-27B Quote
Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
69.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Qwen3.5-35B-A3B Quote
Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
68.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC GPT-5.2 Quote
GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
68.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Claude-Sonnet-4.6 Quote
Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
64.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Qwen3.5-9B Quote
Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
63.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC MedGemma-27B-Text Quote
MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
61.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Fleming-R1-32B Quote
Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
60.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Lingshu-32B Quote
Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
59.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC GPT-OSS-20B Quote
GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
59.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Llama-3.1-70B Quote
Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
55.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC HuatuoGPT-o1-70B Quote
HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
55.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Gemini-2.5-Pro Quote
Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
54.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Fleming-R1-7B Quote
Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
50.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Baichuan-M2-32B Quote
Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
47.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC HuatuoGPT-o1-8B Quote
HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
44.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Lingshu-7B Quote
Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
43.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC HealthGPT-Pro-8B Quote
HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
42.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Hulu-Med-7B Quote
Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
42.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Gemma-3-27B Quote
Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
40.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Llama-3.1-8B Quote
Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
40.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC Gemma-3-12B Quote
Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
39.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC MedGemma-4B Quote
MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
39.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC HealthGPT-Pro-4B Quote
HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
36.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Overall; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy MedGemma-27B-Text Quote
MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
76.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Gemini-3.1-Pro Quote
Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
74.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy GPT-5 Quote
GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
70.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Qwen3.5-35B-A3B Quote
Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
70.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy DeepSeek-V4-Pro Quote
DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
67.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Qwen3.5-27B Quote
Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
67.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Qwen3.5-Plus Quote
Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
66.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Gemini-2.5-Pro Quote
Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
63.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy GPT-5.2 Quote
GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
63.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy HuatuoGPT-o1-70B Quote
HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
63.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Llama-3.1-70B Quote
Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
63.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Claude-Sonnet-4.6 Quote
Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
62.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy GPT-OSS-120B Quote
GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
62.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Lingshu-32B Quote
Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
60.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Fleming-R1-32B Quote
Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
59.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Gemma-3-12B Quote
Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
57.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Llama-3.1-8B Quote
Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
54.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Qwen3.5-9B Quote
Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
54.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Fleming-R1-7B Quote
Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
53.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Gemma-3-27B Quote
Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
53.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy GPT-OSS-20B Quote
GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
52.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Baichuan-M2-32B Quote
Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
49.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy HuatuoGPT-o1-8B Quote
HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
49.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy MedGemma-4B Quote
MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
46.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy HealthGPT-Pro-8B Quote
HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
42.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Lingshu-7B Quote
Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
39.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy HealthGPT-Pro-4B Quote
HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
36.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk activation accuracy Hulu-Med-7B Quote
Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
36.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Activation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Gemini-3.1-Pro Quote
Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
69.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) GPT-5 Quote
GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
63.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Qwen3.5-Plus Quote
Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
57.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) DeepSeek-V4-Pro Quote
DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
56.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) GPT-OSS-120B Quote
GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
56.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Qwen3.5-27B Quote
Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
55.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Qwen3.5-35B-A3B Quote
Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
54.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Lingshu-32B Quote
Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
53.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) GPT-5.2 Quote
GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
52.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) MedGemma-27B-Text Quote
MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
51.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Claude-Sonnet-4.6 Quote
Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
48.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Qwen3.5-9B Quote
Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
48.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Fleming-R1-32B Quote
Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
47.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) GPT-OSS-20B Quote
GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
47.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Llama-3.1-70B Quote
Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
47.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Gemini-2.5-Pro Quote
Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
44.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) HuatuoGPT-o1-70B Quote
HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
42.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Baichuan-M2-32B Quote
Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
37.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Fleming-R1-7B Quote
Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
36.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) HuatuoGPT-o1-8B Quote
HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
35.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Gemma-3-27B Quote
Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
33.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) HealthGPT-Pro-4B Quote
HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
33.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Llama-3.1-8B Quote
Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
33.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Gemma-3-12B Quote
Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
32.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Hulu-Med-7B Quote
Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
32.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) Lingshu-7B Quote
Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
31.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) MedGemma-4B Quote
MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
30.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Counterfactual accuracy (CF) HealthGPT-Pro-8B Quote
HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
28.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Gemini-3.1-Pro Quote
Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
77.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy GPT-5 Quote
GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
61.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy GPT-OSS-120B Quote
GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
55.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Qwen3.5-Plus Quote
Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
55.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Lingshu-32B Quote
Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
54.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Qwen3.5-27B Quote
Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
51.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy DeepSeek-V4-Pro Quote
DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
50.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy GPT-OSS-20B Quote
GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
48.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Qwen3.5-9B Quote
Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
47.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Qwen3.5-35B-A3B Quote
Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
45.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Fleming-R1-32B Quote
Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
44.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy GPT-5.2 Quote
GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
42.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy HealthGPT-Pro-4B Quote
HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
41.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Hulu-Med-7B Quote
Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
38.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Claude-Sonnet-4.6 Quote
Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
35.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Lingshu-7B Quote
Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
32.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Llama-3.1-70B Quote
Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
32.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy MedGemma-27B-Text Quote
MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
32.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Baichuan-M2-32B Quote
Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
30.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Fleming-R1-7B Quote
Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
29.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Gemini-2.5-Pro Quote
Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
27.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy HuatuoGPT-o1-70B Quote
HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
23.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy HealthGPT-Pro-8B Quote
HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
20.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy HuatuoGPT-o1-8B Quote
HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
20.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Gemma-3-27B Quote
Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
10.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Llama-3.1-8B Quote
Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
10.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy MedGemma-4B Quote
MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
10.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Risk deactivation accuracy Gemma-3-12B Quote
Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
7.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Deactivation; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Gemini-3.1-Pro Quote
Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
87.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) GPT-5 Quote
GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
83.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Qwen3.5-Plus Quote
Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
81.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) DeepSeek-V4-Pro Quote
DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
81.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) GPT-OSS-120B Quote
GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
79.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) GPT-5.2 Quote
GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
78.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Qwen3.5-27B Quote
Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
78.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Qwen3.5-35B-A3B Quote
Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
77.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Claude-Sonnet-4.6 Quote
Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
74.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Qwen3.5-9B Quote
Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
73.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Fleming-R1-32B Quote
Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
69.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) MedGemma-27B-Text Quote
MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
67.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) GPT-OSS-20B Quote
GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
66.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) HuatuoGPT-o1-70B Quote
HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
63.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Lingshu-32B Quote
Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
63.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Llama-3.1-70B Quote
Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
61.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Gemini-2.5-Pro Quote
Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
60.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Fleming-R1-7B Quote
Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
59.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Baichuan-M2-32B Quote
Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
54.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) HealthGPT-Pro-8B Quote
HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
51.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Lingshu-7B Quote
Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
50.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) HuatuoGPT-o1-8B Quote
HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
50.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Hulu-Med-7B Quote
Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
48.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) MedGemma-4B Quote
MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
45.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Gemma-3-27B Quote
Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
44.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Llama-3.1-8B Quote
Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
44.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) Gemma-3-12B Quote
Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
44.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Guideline-following accuracy (GF) HealthGPT-Pro-4B Quote
HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
37.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column GF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Gemini-3.1-Pro Quote
Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
48.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy GPT-5 Quote
GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
38.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy DeepSeek-V4-Pro Quote
DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
32.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Qwen3.5-Plus Quote
Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
32.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Qwen3.5-27B Quote
Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
31.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy GPT-OSS-120B Quote
GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
30.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Qwen3.5-35B-A3B Quote
Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
30.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy GPT-5.2 Quote
GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
29.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Lingshu-32B Quote
Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
25.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy GPT-OSS-20B Quote
GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
24.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Qwen3.5-9B Quote
Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
24.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy MedGemma-27B-Text Quote
MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
23.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Claude-Sonnet-4.6 Quote
Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
21.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Fleming-R1-32B Quote
Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
20.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Gemini-2.5-Pro Quote
Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
20.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Llama-3.1-70B Quote
Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
16.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy HuatuoGPT-o1-70B Quote
HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
14.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Hulu-Med-7B Quote
Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
14.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy HuatuoGPT-o1-8B Quote
HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
13.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Fleming-R1-7B Quote
Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
12.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Baichuan-M2-32B Quote
Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
11.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy HealthGPT-Pro-4B Quote
HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
11.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Lingshu-7B Quote
Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
6.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Gemma-3-27B Quote
Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
5.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy HealthGPT-Pro-8B Quote
HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
5.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Llama-3.1-8B Quote
Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
5.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy MedGemma-4B Quote
MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
5.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Linked counterfactual pair accuracy Gemma-3-12B Quote
Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
3.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Pair; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Claude-Sonnet-4.6 Quote
Claude-Sonnet-4.6 | 64.2 | 74.6 | 48.1 | 26.6 | 62.0 | 35.3 | 21.3
26.6 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Claude-Sonnet-4.6, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Qwen3.5-9B Quote
Qwen3.5-9B | 63.8 | 73.9 | 48.1 | 25.9 | 54.9 | 47.1 | 24.7
25.9 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-9B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap GPT-5.2 Quote
GPT-5.2 | 68.1 | 78.2 | 52.5 | 25.7 | 63.4 | 42.6 | 29.2
25.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5.2, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap DeepSeek-V4-Pro Quote
DeepSeek-V4-Pro | 71.7 | 81.3 | 56.8 | 24.5 | 67.6 | 50.0 | 32.6
24.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row DeepSeek-V4-Pro, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Qwen3.5-Plus Quote
Qwen3.5-Plus | 72.4 | 81.7 | 57.9 | 23.8 | 66.2 | 55.9 | 32.6
23.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-Plus, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap HealthGPT-Pro-8B Quote
HealthGPT-Pro-8B | 42.6 | 51.8 | 28.4 | 23.3 | 42.3 | 20.6 | 5.6
23.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-8B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Fleming-R1-7B Quote
Fleming-R1-7B | 50.7 | 59.9 | 36.6 | 23.2 | 53.5 | 29.4 | 12.4
23.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-7B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Qwen3.5-35B-A3B Quote
Qwen3.5-35B-A3B | 68.5 | 77.5 | 54.6 | 22.8 | 70.4 | 45.6 | 30.3
22.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-35B-A3B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap GPT-OSS-120B Quote
GPT-OSS-120B | 70.7 | 79.6 | 56.8 | 22.7 | 62.0 | 55.9 | 30.3
22.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-120B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Qwen3.5-27B Quote
Qwen3.5-27B | 69.4 | 78.2 | 55.7 | 22.4 | 67.6 | 51.5 | 31.5
22.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Qwen3.5-27B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap HuatuoGPT-o1-70B Quote
HuatuoGPT-o1-70B | 55.2 | 63.7 | 42.1 | 21.7 | 63.4 | 23.5 | 14.6
21.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-70B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Fleming-R1-32B Quote
Fleming-R1-32B | 60.6 | 69.0 | 47.5 | 21.5 | 59.2 | 44.1 | 20.2
21.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Fleming-R1-32B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap GPT-5 Quote
GPT-5 | 75.6 | 83.5 | 63.4 | 20.1 | 70.4 | 61.8 | 38.2
20.1 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-5, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap GPT-OSS-20B Quote
GPT-OSS-20B | 59.1 | 66.5 | 47.5 | 19.0 | 52.1 | 48.5 | 24.7
19.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row GPT-OSS-20B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Lingshu-7B Quote
Lingshu-7B | 43.3 | 50.7 | 31.7 | 19.0 | 39.4 | 32.4 | 6.7
19.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-7B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Gemini-3.1-Pro Quote
Gemini-3.1-Pro | 80.7 | 87.7 | 69.9 | 17.7 | 74.6 | 77.9 | 48.3
17.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-3.1-Pro, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Baichuan-M2-32B Quote
Baichuan-M2-32B | 47.8 | 54.6 | 37.2 | 17.4 | 49.3 | 30.9 | 11.2
17.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Baichuan-M2-32B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap MedGemma-27B-Text Quote
MedGemma-27B-Text | 61.2 | 67.6 | 51.4 | 16.2 | 76.1 | 32.4 | 23.6
16.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-27B-Text, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Hulu-Med-7B Quote
Hulu-Med-7B | 42.0 | 48.2 | 32.2 | 16.0 | 36.6 | 38.2 | 14.6
16.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Hulu-Med-7B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Gemini-2.5-Pro Quote
Gemini-2.5-Pro | 54.4 | 60.6 | 44.8 | 15.8 | 63.4 | 27.9 | 20.2
15.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemini-2.5-Pro, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap MedGemma-4B Quote
MedGemma-4B | 39.6 | 45.4 | 30.6 | 14.8 | 46.5 | 10.3 | 5.6
14.8 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row MedGemma-4B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap HuatuoGPT-o1-8B Quote
HuatuoGPT-o1-8B | 44.3 | 50.0 | 35.5 | 14.5 | 49.3 | 20.6 | 13.5
14.5 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HuatuoGPT-o1-8B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Llama-3.1-70B Quote
Llama-3.1-70B | 55.7 | 61.3 | 47.0 | 14.3 | 63.4 | 32.4 | 16.9
14.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-70B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Gemma-3-27B Quote
Gemma-3-27B | 40.3 | 44.7 | 33.3 | 11.4 | 53.5 | 10.3 | 5.6
11.4 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-27B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Gemma-3-12B Quote
Gemma-3-12B | 39.6 | 44.0 | 32.8 | 11.2 | 57.7 | 7.4 | 3.4
11.2 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Gemma-3-12B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Llama-3.1-8B Quote
Llama-3.1-8B | 40.0 | 44.4 | 33.3 | 11.0 | 54.9 | 10.3 | 5.6
11.0 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Llama-3.1-8B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap Lingshu-32B Quote
Lingshu-32B | 59.5 | 63.7 | 53.0 | 10.7 | 60.6 | 54.4 | 25.8
10.7 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row Lingshu-32B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record MedPIC, Published GF-minus-CF accuracy gap HealthGPT-Pro-4B Quote
HealthGPT-Pro-4B | 36.0 | 37.7 | 33.3 | 4.3 | 36.6 | 41.2 | 11.2
4.3 arXiv:2608.03028v1, Section 4, Table 2 (#S4.T2), row HealthGPT-Pro-4B, column Δ_GF−CF; row order: Model | Overall | GF | CF | Δ_GF−CF | Activation | Deactivation | Pair Source of record HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1)Microsoft Research (Liu, Zhang, Qin et al.), third-party, 30 Jun 20269 results
Open the document arxiv.org, retrieved 7 Sep 2026
Board Row Printed Where Role HealthAgentBench Codex (GPT 5.5) Quote
Codex GPT-5.5 ... 42% $2.8 15 min
42% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Copilot (Opus 4.8) Quote
Copilot Opus 4.8 ... 36% $3.1 18 min
36% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Copilot (GPT 5.5) Quote
Copilot GPT-5.5 ... 35% $2.6 18 min
35% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Claude Code (Opus 4.8) Quote
Claude Code Opus 4.8 ... 32% $4.0 16 min
32% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Codex (GPT 5.4) Quote
Codex GPT-5.4 ... 28% $1.3 18 min
28% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Claude Code (Opus 4.7) Quote
Claude Code Opus 4.7 ... 27% $4.8 16 min
27% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Claude Code (Opus 4.6) Quote
Claude Code Opus 4.6 ... 19% $4.1 22 min
19% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Claude Code (Sonnet 4.6) Quote
Claude Code Sonnet 4.6 ... 17% $2.9 24 min
17% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it HealthAgentBench Codex (GPT 5.4 Mini) Quote
Codex GPT-5.4-mini ... 16% $0.6 16 min
16% p. 11, Figure 4 (pooled task success rate, ten agents) Also prints it EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning (arXiv 2606.23301)Zhejiang University / Ant Group (Qiao, Liu, Chu et al.), third-party, 22 Jun 20266 results
Open the document arxiv.org, retrieved 7 Sep 2026
Board Row Printed Where Role EHR-Complex GPT-5.4 (high reasoning) 0.65 arXiv abs page (landing page for the PDF) Also prints it EHR-Complex Gemini 3.1 Pro 0.63 arXiv abs page (landing page for the PDF) Also prints it EHR-Complex Kimi-K2.5 0.62 arXiv abs page (landing page for the PDF) Also prints it EHR-Complex Qwen3.5-397B 0.62 arXiv abs page (landing page for the PDF) Also prints it EHR-Complex GPT-5.4 (low reasoning) 0.58 arXiv abs page (landing page for the PDF) Also prints it EHR-Complex Claude Sonnet 4.6 0.36 arXiv abs page (landing page for the PDF) Also prints it EHR-Complex: Benchmarking Medical Agents for Complex Clinical Reasoning (arXiv 2606.23301v1 PDF)Zhejiang University / Ant Group (Qiao, Liu, Chu et al.), third-party, 22 Jun 202618 results
Open the document arxiv.org, retrieved 7 Sep 2026
Board Row Printed Where Role EHR-Complex GPT-5.4 (high reasoning) Quote
GPT-5.4 (high) 0.85 0.36 0.78 0.34 0.8 0.37 0.93 0.76 0.91 0.49 0.84 0.36 0.65
0.65 p. 15, Table 10 (Strong commercial model results), Avg. column Source of record EHR-Complex Gemini 3.1 Pro Quote
Gemini 3.1 Pro 0.87 0.32 0.67 0.38 0.84 0.37 0.92 0.63 0.92 0.44 0.83 0.34 0.63
0.63 p. 15, Table 10 (Strong commercial model results), Avg. column Source of record EHR-Complex Kimi-K2.5 Quote
Kimi-K2.5 0.89 0.34 0.8 0.27 0.73 0.35 0.92 0.72 0.89 0.42 0.84 0.29 0.62
0.62 p. 6, Table 3 (Evaluation Results on the EHR-Complex Test Set), Avg. column Source of record EHR-Complex Qwen3.5-397B Quote
Qwen3.5-397B 0.88 0.39 0.72 0.3 0.8 0.38 0.91 0.63 0.91 0.45 0.78 0.32 0.62
0.62 p. 6, Table 3 (Evaluation Results on the EHR-Complex Test Set), Avg. column Source of record EHR-Complex DeepSeek-V3.2-Exp Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. DeepSeek-V3.2-Exp | 0.83 | 0.36 | 0.75 | 0.27 | 0.7 | 0.3 | 0.88 | 0.65 | 0.89 | 0.44 | 0.77 | 0.26 | 0.59
0.59 arXiv HTML 2606.23301v1, Table 3, DeepSeek-V3.2-Exp row, final Avg. column (PDF p. 6). Source of record EHR-Complex GPT-5.4 (low reasoning) Quote
GPT-5.4 (low) 0.81 0.32 0.65 0.26 0.73 0.31 0.88 0.66 0.86 0.43 0.8 0.29 0.58
0.58 p. 15, Table 10 (Strong commercial model results), Avg. column Source of record EHR-Complex DeepSeek-V3.1 Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. DeepSeek-V3.1 | 0.81 | 0.33 | 0.7 | 0.27 | 0.65 | 0.24 | 0.89 | 0.68 | 0.82 | 0.39 | 0.72 | 0.21 | 0.56
0.56 arXiv HTML 2606.23301v1, Table 3, DeepSeek-V3.1 row, final Avg. column (PDF p. 6). Source of record EHR-Complex Qwen3-32B-SFT Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. Qwen3-32B-SFT | 0.83 | 0.36 | 0.55 | 0.29 | 0.72 | 0.3 | 0.86 | 0.55 | 0.8 | 0.34 | 0.69 | 0.26 | 0.55
0.55 arXiv HTML 2606.23301v1, Table 3, Qwen3-32B-SFT row, final Avg. column (PDF p. 6). Source of record EHR-Complex Qwen3-235B Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. Qwen3-235B | 0.75 | 0.27 | 0.65 | 0.26 | 0.72 | 0.25 | 0.91 | 0.55 | 0.8 | 0.36 | 0.7 | 0.21 | 0.53
0.53 arXiv HTML 2606.23301v1, Table 3, Qwen3-235B row, final Avg. column (PDF p. 6). Source of record EHR-Complex GPT-4.1 mini Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. GPT-4.1 mini | 0.75 | 0.27 | 0.6 | 0.22 | 0.67 | 0.21 | 0.88 | 0.6 | 0.65 | 0.3 | 0.59 | 0.17 | 0.49
0.49 arXiv HTML 2606.23301v1, Table 3, GPT-4.1 mini row, final Avg. column (PDF p. 6). Source of record EHR-Complex GPT-4.1 Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. GPT-4.1 | 0.76 | 0.21 | 0.55 | 0.16 | 0.59 | 0.17 | 0.89 | 0.5 | 0.72 | 0.26 | 0.63 | 0.18 | 0.47
0.47 arXiv HTML 2606.23301v1, Table 3, GPT-4.1 row, final Avg. column (PDF p. 6). Source of record EHR-Complex Qwen3-14B-SFT Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. Qwen3-14B-SFT | 0.69 | 0.22 | 0.43 | 0.15 | 0.6 | 0.19 | 0.84 | 0.51 | 0.74 | 0.22 | 0.63 | 0.18 | 0.45
0.45 arXiv HTML 2606.23301v1, Table 3, Qwen3-14B-SFT row, final Avg. column (PDF p. 6). Source of record EHR-Complex Claude Sonnet 4.6 Quote
Claude Sonnet 4.6 0.59 0.21 0.45 0.12 0.45 0.12 0.83 0.2 0.49 0.33 0.4 0.11 0.36
0.36 p. 15, Table 10 (Strong commercial model results), Avg. column Source of record EHR-Complex Qwen3-32B Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. Qwen3-32B | 0.51 | 0.2 | 0.3 | 0.24 | 0.47 | 0.2 | 0.73 | 0.35 | 0.5 | 0.21 | 0.42 | 0.15 | 0.36
0.36 arXiv HTML 2606.23301v1, Table 3, Qwen3-32B row, final Avg. column (PDF p. 6). Source of record EHR-Complex Gemini 2.5 Pro Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. Gemini 2.5 Pro | 0.44 | 0.19 | 0.18 | 0.2 | 0.43 | 0.15 | 0.59 | 0.38 | 0.38 | 0.27 | 0.34 | 0.15 | 0.31
0.31 arXiv HTML 2606.23301v1, Table 3, Gemini 2.5 Pro row, final Avg. column (PDF p. 6). Source of record EHR-Complex GPT-4o Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. GPT-4o | 0.52 | 0.1 | 0.45 | 0.26 | 0.33 | 0.07 | 0.53 | 0.33 | 0.46 | 0.19 | 0.38 | 0.08 | 0.31
0.31 arXiv HTML 2606.23301v1, Table 3, GPT-4o row, final Avg. column (PDF p. 6). Source of record EHR-Complex Qwen3-14B Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. Qwen3-14B | 0.44 | 0.15 | 0.21 | 0.18 | 0.43 | 0.15 | 0.63 | 0.33 | 0.4 | 0.19 | 0.39 | 0.1 | 0.3
0.3 arXiv HTML 2606.23301v1, Table 3, Qwen3-14B row, final Avg. column (PDF p. 6). Source of record EHR-Complex Qwen3-4B Quote
Model | Demographics | Vitals | Medications | Cost | Labs | Diagnoses | Avg. Qwen3-4B | 0.24 | 0.07 | 0.28 | 0.11 | 0.18 | 0.01 | 0.48 | 0.12 | 0.17 | 0.06 | 0.16 | 0.03 | 0.16
0.16 arXiv HTML 2606.23301v1, Table 3, Qwen3-4B row, final Avg. column (PDF p. 6). Source of record PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments (arXiv 2605.02240)Stanford University (HealthRex; Liu, Chen et al.), third-party, 4 May 20268 results
Open the document arxiv.org, retrieved 7 Sep 2026
Board Row Printed Where Role PhysicianBench GPT-5.5 46.3 ± 1.2 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench Claude Opus 4.6 31.7 ± 2.3 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench Claude Opus 4.7 29.3 ± 2.5 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench GPT-5.4 27.7 ± 1.5 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench Claude Sonnet 4.6 23.0 ± 2.6 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench Kimi-K2.6 17.0 ± 2.6 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench Qwen3.6-Plus 13.7 ± 4.0 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench Gemini Pro 3.1 6.0 ± 1.0 arXiv abs page (landing page for the PDF) Also prints it PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments (arXiv 2605.02240v1 PDF)Stanford University (HealthRex; Liu, Chen et al.), third-party, 4 May 202612 results
Open the document arxiv.org, retrieved 7 Sep 2026
Board Row Printed Where Role PhysicianBench GPT-5.5 Quote
GPT-5.5 46.3 ± 1.2 57.4 28.0 41.9
46.3 ± 1.2 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench Claude Opus 4.6 Quote
Claude Opus 4.6 31.7 ± 2.3 41.5 18.0 25.2
31.7 ± 2.3 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench Claude Opus 4.7 Quote
Claude Opus 4.7 29.3 ± 2.5 37.9 18.0 16.2
29.3 ± 2.5 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench GPT-5.4 Quote
GPT-5.4 27.7 ± 1.5 37.7 13.0 39.8
27.7 ± 1.5 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench Claude Sonnet 4.6 Quote
Claude Sonnet 4.6 23.0 ± 2.6 33.2 9.0 22.3
23.0 ± 2.6 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench DeepSeek V4-Pro Quote
Model | Pass@1 | Pass@3 | Pass^3 | #Turns DeepSeek V4-Pro | 18.7 ± 2.9 | 27.9 | 6.0 | 35.3
18.7 ± 2.9 arXiv HTML 2605.02240v1, Section 5.2, Table 2; DeepSeek V4-Pro row, Pass@1 column (PDF p. 8). Source of record PhysicianBench Kimi-K2.6 Quote
Kimi-K2.6 17.0 ± 2.6 26.3 5.0 42.4
17.0 ± 2.6 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench MiMo-v2.5-Pro Quote
Model | Pass@1 | Pass@3 | Pass^3 | #Turns MiMo-v2.5-Pro | 16.7 ± 4.0 | 23.6 | 6.0 | 29.5
16.7 ± 4.0 arXiv HTML 2605.02240v1, Section 5.2, Table 2; MiMo-v2.5-Pro row, Pass@1 column (PDF p. 8). Source of record PhysicianBench Qwen3.6-Plus Quote
Qwen3.6-Plus 13.7 ± 4.0 22.6 2.0 28.0
13.7 ± 4.0 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench MiniMax M2.7 Quote
Model | Pass@1 | Pass@3 | Pass^3 | #Turns MiniMax M2.7 | 8.7 ± 1.2 | 15.9 | 1.0 | 29.7
8.7 ± 1.2 arXiv HTML 2605.02240v1, Section 5.2, Table 2; MiniMax M2.7 row, Pass@1 column (PDF p. 8). Source of record PhysicianBench Gemini Pro 3.1 Quote
Gemini Pro 3.1 6.0 ± 1.0 9.3 3.0 30.4
6.0 ± 1.0 p. 8, Table 2 (columns Pass@1, Pass@3, Pass^3, #Turns) Source of record PhysicianBench Grok-4.20 Quote
Grok-4.20 5.3 ± 3.2 9.7 1.0 16.7
5.3 ± 3.2 p. 8, Table 2 (Proprietary Models block) Source of record HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks (arXiv 2604.09937v1)Bedi, Welch, Steinberg et al. (Stanford University / Kinetic Systems / Stanford Health Care), third-party, 10 Apr 20267 results
Open the document arxiv.org, retrieved 7 Sep 2026
Board Row Printed Where Role HealthAdminBench Claude Opus 4.6 (computer-use agent) Quote
Claude Opus 4.6 CUA | 36.3%
36.3% p. 8, Figure 3(a), Task Success Rate, bar "Claude Opus 4.6 CUA"; section 4, p. 7 gives screenshot-only, Task Description + Portal Guidance; arXiv 2604.09937v1 dated 2026-04-10; checked 2026-09-30. Source of record HealthAdminBench GPT-5.4 (computer-use agent) Quote
GPT-5.4 CUA | 26.7%
26.7% p. 8, Figure 3(a), Task Success Rate, bar "GPT-5.4 CUA"; section 4, p. 7 gives screenshot-only, Task Description + Portal Guidance; arXiv 2604.09937v1 dated 2026-04-10; checked 2026-09-30. Source of record HealthAdminBench Kimi K2.5 Quote
Kimi K2.5 | 15.6%
15.6% p. 8, Figure 3(a), Task Success Rate, bar "Kimi K2.5"; section 4, p. 7 gives screenshot-only, Task Description + Portal Guidance; arXiv 2604.09937v1 dated 2026-04-10; checked 2026-09-30. Source of record HealthAdminBench Claude Opus 4.6 (standardized harness) Quote
Claude Opus 4.6 | 14.8%
14.8% p. 8, Figure 3(a), Task Success Rate, bar "Claude Opus 4.6"; section 4, p. 7 gives screenshot-only, Task Description + Portal Guidance; arXiv 2604.09937v1 dated 2026-04-10; checked 2026-09-30. Source of record HealthAdminBench Qwen 3.5 Quote
Qwen 3.5 | 13.3%
13.3% p. 8, Figure 3(a), Task Success Rate, bar "Qwen 3.5"; section 4, p. 7 gives screenshot-only, Task Description + Portal Guidance; arXiv 2604.09937v1 dated 2026-04-10; checked 2026-09-30. Source of record HealthAdminBench Gemini 3.1 Pro Quote
Gemini 3.1 Pro | 11.9%
11.9% p. 8, Figure 3(a), Task Success Rate, bar "Gemini 3.1 Pro"; section 4, p. 7 gives screenshot-only, Task Description + Portal Guidance; arXiv 2604.09937v1 dated 2026-04-10; checked 2026-09-30. Source of record HealthAdminBench GPT-5.4 (standardized harness) Quote
GPT-5.4 | 5.9%
5.9% p. 8, Figure 3(a), Task Success Rate, bar "GPT-5.4"; section 4, p. 7 gives screenshot-only, Task Description + Portal Guidance; arXiv 2604.09937v1 dated 2026-04-10; checked 2026-09-30. Source of record Gemma 4 Technical ReportGoogle DeepMind, third-party, date not stated1 result
Open the document arxiv.org, retrieved 7 Sep 2026
Board Row Printed Where Role MedXpertQA (MM) Gemma 4 12B Quote
MedXPertQA MM 61.3 58.1 48.7 28.7 23.5 -
48.7 Gemma 4 Technical Report p. 6, Table 6 (vision benchmarks, thinking), row MedXPertQA MM, column 12B Also prints it
System card
Claude Haiku 5.5 System CardAnthropic, first-party, 7 Oct 202610 results
Open the document www.anthropic.com, retrieved 12 Oct 2026
Board Row Printed Where Role HealthBench Professional GPT-6 Luna (Anthropic run) Quote
HealthBench Professional panel: 65.5% (GPT-6 Luna (OpenAI)); caption: GPT-6 Luna's scores come from our own runs through OpenAI's public API and are not OpenAI's published figures.
65.5% p. 132, Figure 8.11.1.A, HealthBench Professional panel, GPT-6 Luna (OpenAI) diamond label (visually read printed label); Section 8.11.1 text on pp. 131-132 Source of record HealthBench Professional Claude Haiku 5.5 Quote
Max effort scored highest, at 64.8% (71.0% raw); thirteen repeat runs at max on October 4, the day of the lower levels, averaged 64.2%.
64.8% p. 133, Section 8.11.3; same value in p. 111, Table 8.1.A, row HealthBench Professional (Length-adjusted), column Haiku 5.5 (printed 64.8) and p. 132, Figure 8.11.1.A Source of record HealthBench Professional Claude Haiku 4.5 Quote
Evaluation | Haiku 5.5 | Haiku 4.5 | Sonnet 5.5 | GPT-6 Luna HealthBench Professional (Length-adjusted) | 64.8 | 32.2 | 69.2 | –
32.2 p. 111, Table 8.1.A, row HealthBench Professional (Length-adjusted), column Haiku 4.5 Source of record PhysicianBench Claude Haiku 5.5 (max) Quote
PhysicianBench is a public benchmark of 100 physician tasks carried out in an electronic health record (EHR). Claude Haiku 5.5 passed 43.0% of attempts (five attempts on each task).
43.0% pp. 132-133, Section 8.11.2, Figure 8.11.2.A caption and paragraph; also p. 133, Section 8.11.3 Source of record PhysicianBench Claude Haiku 5.5 (xhigh) Quote
On PhysicianBench (five attempts per task at each level), Haiku 5.5 passed 17.8% of attempts at low, 25.2% at medium, 31.6% at high, 35.8% at xhigh and 43.0% at max.
35.8% p. 133, Section 8.11.3; p. 134, Figure 8.11.3.A PhysicianBench panel Source of record PhysicianBench GPT-6 Luna (Anthropic run) Quote
This was 9.4 points above GPT-6 Luna at 33.6% (95% confidence interval 2.6 to 16.2 points, paired by task)
33.6% p. 133, Section 8.11.2; p. 132, Figure 8.11.2.A GPT-6 Luna bar; p. 131-132, Section 8.11.1 says GPT-6 Luna scores come from Anthropic's own runs Source of record PhysicianBench Claude Haiku 5.5 (high) Quote
On PhysicianBench (five attempts per task at each level), Haiku 5.5 passed 17.8% of attempts at low, 25.2% at medium, 31.6% at high, 35.8% at xhigh and 43.0% at max.
31.6% p. 133, Section 8.11.3; p. 134, Figure 8.11.3.A PhysicianBench panel Source of record PhysicianBench Claude Haiku 5.5 (medium) Quote
On PhysicianBench (five attempts per task at each level), Haiku 5.5 passed 17.8% of attempts at low, 25.2% at medium, 31.6% at high, 35.8% at xhigh and 43.0% at max.
25.2% p. 133, Section 8.11.3; p. 134, Figure 8.11.3.A PhysicianBench panel Source of record PhysicianBench Claude Haiku 5.5 (low) Quote
On PhysicianBench (five attempts per task at each level), Haiku 5.5 passed 17.8% of attempts at low, 25.2% at medium, 31.6% at high, 35.8% at xhigh and 43.0% at max.
17.8% p. 133, Section 8.11.3; p. 134, Figure 8.11.3.A PhysicianBench panel Source of record PhysicianBench Claude Haiku 4.5 (extended thinking) Quote
Claude Haiku 4.5 | 1.4%
1.4% p. 132, Figure 8.11.2.A PhysicianBench pass rate, Claude Haiku 4.5 bar label (visually read printed label; not repeated in the text) Source of record Addendum to GPT-6 Astra System Card: GPT-6.1 SolOpenAI, first-party, 29 Sep 20261 result
Open the document deploymentsafety.openai.com, retrieved 6 Oct 2026
Board Row Printed Where Role HealthBench Professional GPT-6.1 Sol Quote
EVALUATION | GPT-5.5 | GPT-5.6 SOL | GPT-5.6 TERRA | GPT-5.6 LUNA | GPT-6 ASTRA | GPT-6 SOL | GPT-6 LUNA | GPT-6.1 SOL HealthBench Professional length-adjusted | 51.8 (57.2, 3818) | 60.5 (64.1, 3228) | 57.7 (62.4, 3618) | 55.7 (59.8, 3389) | 64.7 (68.2, 3185) | 60.8 (59.5, 1573) | 60.8 (61.2, 2119) | 64.2 (67.2, 3038)
64.2 Section 5.1, Table 7, HealthBench Professional length-adjusted, GPT-6.1 Sol column Source of record Claude Sonnet 5.5 System CardAnthropic, first-party, 28 Sep 202611 results
Open the document www.anthropic.com
Board Row Printed Where Role HealthBench Professional GPT-6 Astra (Anthropic run) Quote
On HealthBench Professional at max effort, length adjustment changes the ranking. After: GPT-6 Astra (70.3%) > Claude Sonnet 5.5 (69.2%) > Claude Opus 5.5 (65.6%) > Claude Fable 5.1 (62.1%).
70.3% p. 138, paragraph above Figure 8.15.B, HealthBench Professional after length adjustment. Source of record HealthBench Professional Claude Sonnet 5.5 Quote
On HealthBench Professional at max effort, length adjustment changes the ranking. After: GPT-6 Astra (70.3%) > Claude Sonnet 5.5 (69.2%) > Claude Opus 5.5 (65.6%) > Claude Fable 5.1 (62.1%).
69.2% p. 138, Section 8.15.2 and paragraph above Figure 8.15.B. Source of record PhysicianBench Claude Opus 5.5 (max) Quote
PhysicianBench is a public benchmark of 100 physician tasks carried out in an EHR. At max effort, Claude Sonnet 5.5 passes 63.2% of attempts, well above Claude Sonnet 5 (37.4%), level with Claude Fable 5.1 (61.0%), about 6 points above Claude Opus 5 (57.6%) and about 5 points below Claude Opus 5.5 (68.4%).
68.4% pp. 138–139, Section 8.15.3; PhysicianBench pass@1, Claude Opus 5.5 (max). Source of record PhysicianBench Claude Sonnet 5.5 (max) Quote
PhysicianBench is a public benchmark of 100 physician tasks carried out in an EHR. At max effort, Claude Sonnet 5.5 passes 63.2% of attempts, well above Claude Sonnet 5 (37.4%), level with Claude Fable 5.1 (61.0%), about 6 points above Claude Opus 5 (57.6%) and about 5 points below Claude Opus 5.5 (68.4%).
63.2% pp. 138–139, Section 8.15.3; PhysicianBench pass@1, Claude Sonnet 5.5 (max). Source of record PhysicianBench Claude Fable 5.1 (max) Quote
PhysicianBench is a public benchmark of 100 physician tasks carried out in an EHR. At max effort, Claude Sonnet 5.5 passes 63.2% of attempts, well above Claude Sonnet 5 (37.4%), level with Claude Fable 5.1 (61.0%), about 6 points above Claude Opus 5 (57.6%) and about 5 points below Claude Opus 5.5 (68.4%).
61.0% pp. 138–139, Section 8.15.3; PhysicianBench pass@1, Claude Fable 5.1 (max). Source of record PhysicianBench Claude Opus 5 (max) Quote
PhysicianBench is a public benchmark of 100 physician tasks carried out in an EHR. At max effort, Claude Sonnet 5.5 passes 63.2% of attempts, well above Claude Sonnet 5 (37.4%), level with Claude Fable 5.1 (61.0%), about 6 points above Claude Opus 5 (57.6%) and about 5 points below Claude Opus 5.5 (68.4%).
57.6% pp. 138–139, Section 8.15.3; PhysicianBench pass@1, Claude Opus 5 (max). Source of record PhysicianBench Claude Sonnet 5.5 (xhigh) Quote
Sonnet 5.5 | PhysicianBench (pass@1) | xhigh 56.4%
56.4% p. 138, Figure 8.15.B, right panel PhysicianBench (pass@1); orange Sonnet 5.5 xhigh label (visually read printed labels). Source of record PhysicianBench Claude Sonnet 5.5 (high) Quote
Sonnet 5.5 | PhysicianBench (pass@1) | high 47.6%
47.6% p. 138, Figure 8.15.B, right panel PhysicianBench (pass@1); orange Sonnet 5.5 high label (visually read printed labels). Source of record PhysicianBench Claude Sonnet 5 (max) Quote
PhysicianBench is a public benchmark of 100 physician tasks carried out in an EHR. At max effort, Claude Sonnet 5.5 passes 63.2% of attempts, well above Claude Sonnet 5 (37.4%), level with Claude Fable 5.1 (61.0%), about 6 points above Claude Opus 5 (57.6%) and about 5 points below Claude Opus 5.5 (68.4%).
37.4% pp. 138–139, Section 8.15.3; PhysicianBench pass@1, Claude Sonnet 5 (max). Source of record PhysicianBench Claude Sonnet 5.5 (medium) Quote
Sonnet 5.5 | PhysicianBench (pass@1) | medium 30.0%
30.0% p. 138, Figure 8.15.B, right panel PhysicianBench (pass@1); orange Sonnet 5.5 medium label (visually read printed labels). Source of record PhysicianBench Claude Sonnet 5.5 (low) Quote
Sonnet 5.5 | PhysicianBench (pass@1) | low 27.2%
27.2% p. 138, Figure 8.15.B, right panel PhysicianBench (pass@1); orange Sonnet 5.5 low label (visually read printed labels). Source of record Claude Opus 5.5 System CardAnthropic, first-party, 22 Sep 20261 result
Open the document www.anthropic.com
Board Row Printed Where Role HealthBench Professional Claude Opus 5.5 Quote
8.15.2 HealthBench Professional results length adjustment, which penalizes verbose model responses, Claude Opus 5.5 achieved a score of 65.6%.
65.6% pp. 213–214, Section 8.15.2, paragraph and Figure 8.15.2.A. Source of record GPT-6 Astra System CardOpenAI, first-party, 3 Sep 20261 result
Open the document deploymentsafety.openai.com, retrieved 8 Sep 2026
Board Row Printed Where Role HealthBench Professional GPT-6 Astra Quote
Astra has a length-adjusted HealthBench Professional score of 63.4 (+2.9 relative to GPT-5.6 Sol), HealthBench score of 58.1 (+1.1), HealthBench Hard score of 36.3 (+3.2), and HealthBench Consensus score of 95.8 (+0.3).
63.4 p. 19, sec. 6.1 (Table 6 cell, column 'gpt-6 Astra': 63.4 (69.5, 4097)) Prints a different number GPT-6 Astra System Card - HealthBench (Deployment Safety Hub)OpenAI, first-party, 3 Sep 20264 results
Open the document deploymentsafety.openai.com, retrieved 8 Sep 2026
Board Row Printed Where Role HealthBench Professional GPT-6 Astra Quote
Evaluation | GPT-5.5 | GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | GPT-6 Astra | GPT-6 Sol | GPT-6 Luna HealthBench Professional length-adjusted | 51.8 (57.2, 3818) | 60.5 (64.1, 3228) | 57.7 (62.4, 3618) | 55.7 (59.8, 3389) | 64.7 (68.2, 3185) | 60.8 (59.5, 1573) | 60.8 (61.2, 2119)
64.7 Section 11.4.1, Table 29, HealthBench Professional length-adjusted, GPT-6 Astra column; September 22 correction. Source of record HealthBench Professional GPT-6 Astra Quote
Astra has a length-adjusted HealthBench Professional score of 63.4 (+2.9 relative to GPT-5.6 Sol)
63.4 section 6.1, HTML rendering of the same card Mirror HealthBench Professional GPT-6 Luna Quote
Evaluation | GPT-5.5 | GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | GPT-6 Astra | GPT-6 Sol | GPT-6 Luna HealthBench Professional length-adjusted | 51.8 (57.2, 3818) | 60.5 (64.1, 3228) | 57.7 (62.4, 3618) | 55.7 (59.8, 3389) | 64.7 (68.2, 3185) | 60.8 (59.5, 1573) | 60.8 (61.2, 2119)
60.8 Section 11.4.1, Table 29, HealthBench Professional length-adjusted, GPT-6 Luna column; September 22 correction. Source of record HealthBench Professional GPT-6 Sol Quote
Evaluation | GPT-5.5 | GPT-5.6 Sol | GPT-5.6 Terra | GPT-5.6 Luna | GPT-6 Astra | GPT-6 Sol | GPT-6 Luna HealthBench Professional length-adjusted | 51.8 (57.2, 3818) | 60.5 (64.1, 3228) | 57.7 (62.4, 3618) | 55.7 (59.8, 3389) | 64.7 (68.2, 3185) | 60.8 (59.5, 1573) | 60.8 (61.2, 2119)
60.8 Section 11.4.1, Table 29, HealthBench Professional length-adjusted, GPT-6 Sol column; September 22 correction. Source of record Claude Fable 5.1 and Claude Mythos 5.1 System CardAnthropic, first-party, 1 Sep 20264 results
Open the document www.anthropic.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional Claude Fable 5 Quote
HealthBench Professional 62.1% 63.3% 59.8% –
63.3% p. 167, Table 8.1.A, column 'Claude Fable 5/Mythos 5' Prints a different number HealthBench Professional Claude Fable 5 (September card) Quote
HealthBench Professional | Length-adjusted score | Claude Fable 5 | 63.3%
63.3% p. 199, Figure 8.17.2.A, Claude Fable 5 length-adjusted bar (visually read printed labels). Source of record HealthBench Professional Claude Fable 5.1 Quote
On HealthBench Professional, Claude Fable 5.1 achieved a raw score of 74.2%, ahead of Claude Opus 5 at 73.4%, Fable 5 at 68.9%, and Claude Sonnet 5 at 62.4%. After length adjustment, which penalizes verbose model responses, Fable 5.1 achieved a score of 62.1%.
62.1% p. 199, sec. 8.17.2 (same figure printed as 62.1% in Table 8.1.A, p. 167) Source of record HealthBench Professional Claude Opus 5 Quote
HealthBench Professional 62.1% 63.3% 59.8% –
59.8% p. 167, Table 8.1.A, column 'Claude Opus 5'; raw 73.4% also reprinted on p. 199, sec. 8.17.2 Also prints it GPT-5.6 - August Updates (system card addendum)OpenAI, first-party, 6 Aug 20263 results
Open the document cdn.openai.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional GPT-5.6 Sol (August) Quote
HealthBench Professional 32.9 (33.8, 2,285) 38.4 (40.7, 2,775) 54.0 (56.6, 2,894) 44.1 (46.8, 2,920)
54.0 p. 11, section 5.1 HealthBench, table "Reported as length-adjusted score (unadjusted, mean response length in characters)", column GPT-5.6 Sol (August) Source of record HealthBench Professional GPT-5.6 Luna (August) Quote
HealthBench Professional 32.9 (33.8, 2,285) 38.4 (40.7, 2,775) 54.0 (56.6, 2,894) 44.1 (46.8, 2,920)
44.1 p. 11, section 5.1 HealthBench, table "Reported as length-adjusted score (unadjusted, mean response length in characters)", column GPT-5.6 Luna (August) Source of record HealthBench Professional GPT-5.5 Instant Quote
HealthBench Professional 32.9 (33.8, 2,285) 38.4 (40.7, 2,775) 54.0 (56.6, 2,894) 44.1 (46.8, 2,920)
38.4 p. 11, section 5.1 HealthBench, table "Reported as length-adjusted score (unadjusted, mean response length in characters)", column GPT-5.5 Instant Also prints it System Card: Claude Opus 5Anthropic, first-party, 24 Jul 20264 results
Open the document www.anthropic.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional Claude Fable 5 Quote
HealthBench Professional 59.8 57.4 66.011 60.5
66.0 p. 152, Table 8.1.A, column Fable 5 (footnote 11: Mythos 5) Also prints it HealthBench Professional Claude Opus 5 Quote
Claude Opus 5 achieved a raw score of 73.4%, which is the highest amongst all Claude models, ahead of Claude Mythos 5 at 70.3%, Claude Opus 4.8 at 60.3%, and Claude Sonnet 5 at 62.4%. After length adjustment, which penalizes verbose model responses, Claude Opus 5 achieved a score of 59.8%.
59.8% p. 189, section 8.15.2 HealthBench Professional results; also Table 8.1.A p. 152 ('HealthBench Professional 59.8 ...') Source of record HealthBench Professional Claude Sonnet 5 Quote
Figure 8.15.2.A bar labels: Claude Sonnet 5 62.4% raw / 57.8% length-adjusted (prose: 'Claude Sonnet 5 at 62.4%')
57.8% p. 189, section 8.15.2, Figure 8.15.2.A Also prints it HealthBench Professional Claude Opus 4.8 Quote
HealthBench Professional 59.8 57.4 66.011 60.5
57.4 p. 152, Table 8.1.A, column Opus 4.8 Prints a different number GPT-5.6 System CardOpenAI, first-party, 9 Jul 20268 results
Open the document deploymentsafety.openai.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional GPT-5.6 Sol Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
60.5 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.6-SOL Source of record HealthBench Professional GPT-5.6 Terra Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
57.7 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.6-TERRA Source of record HealthBench Professional GPT-5.6 Luna Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
55.7 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.6-LUNA Source of record HealthBench Professional GPT-5.5 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
51.8 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.5 Source of record HealthBench Professional GPT-5.4 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
48.1 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.4 Source of record HealthBench Professional GPT-5 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
46.2 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5 Source of record HealthBench Professional GPT-5.2 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
45.9 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.2 Source of record HealthBench Professional GPT-5.1 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
39.6 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.1 Source of record System Card: Claude Sonnet 5Anthropic, first-party, 30 Jun 20263 results
Open the document www.anthropic.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional Claude Sonnet 5 Quote
HealthBench Professional 57.8 44.2 51.8 -
57.8 p. 115, Table 8.1.A, row HealthBench Professional, column Claude Sonnet 5; Figure 8.12.2.A p. 139 Source of record HealthBench Professional Claude Opus 4.8 (Opus 4.8 grader) Quote
HealthBench Professional | Length-adjusted score (%) | Claude Opus 4.8 | 57.4%
57.4% p. 139, Figure 8.12.2.A, Claude Opus 4.8 bar (visually read printed labels). Source of record HealthBench Professional Claude Sonnet 4.6 Quote
HealthBench Professional 57.8 44.2 51.8 -
44.2 p. 115, Table 8.1.A, row HealthBench Professional, column Claude Sonnet 4.6 Source of record GPT-5.6 Preview System CardOpenAI, first-party, 26 Jun 20268 results
Open the document deploymentsafety.openai.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional GPT-5.6 Sol Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
60.5 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5.6-SOL Also prints it HealthBench Professional GPT-5.6 Terra Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
57.7 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5.6-TERRA Also prints it HealthBench Professional GPT-5.6 Luna Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
55.7 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5.6-LUNA Also prints it HealthBench Professional GPT-5.5 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
51.8 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5.5 Also prints it HealthBench Professional GPT-5.4 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
48.1 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5.4 Also prints it HealthBench Professional GPT-5 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
46.2 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5 Also prints it HealthBench Professional GPT-5.2 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
45.9 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5.2 Also prints it HealthBench Professional GPT-5.1 Quote
HealthBench Professional length-adjusted 46.2 (51.0, 3616) 39.6 (48.0, 4863) 45.9 (50.0, 3400) 48.1 (51.9, 3308) 51.8 (57.2, 3818) 60.5 (64.1, 3228) 57.7 (62.4, 3618) 55.7 (59.8, 3389)
39.6 Section 5.1 HealthBench, Table 6 (reported as length-adjusted score (unadjusted, mean response length in characters)) (preview card, published 2026-06-26), column GPT-5.1 Also prints it Claude Fable 5 and Claude Mythos 5 System CardAnthropic, first-party, 9 Jun 20262 results
Open the document anthropic.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional Claude Fable 5 Quote
HealthBench Professional 66.0 - 64.7 56.9 51.8 -
66.0 p. 252, Table 8.1.A, row HealthBench Professional, column Mythos 5 (Fable 5 column '-'); Figure 8.18.2.A p. 298 bar label 66.0% on Claude Mythos 5 Source of record HealthBench Professional Claude Opus 4.8 Quote
HealthBench Professional 66.0 - 64.7 56.9 51.8 -
56.9 p. 252, Table 8.1.A, column Opus 4.8 Prints a different number System Card: Claude Opus 4.8Anthropic, first-party, 28 May 20263 results
Open the document www.anthropic.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional Claude Opus 4.8 Quote
Claude Opus 4.8 scores 55.8%, a meaningful improvement over Claude Opus 4.7 at 51.9% and Claude Sonnet 4.6 at 41.7%.
55.8% p. 228, section 8.14.1 HealthBench Professional; Figure 8.14.A p. 229 Source of record HealthBench Professional Claude Opus 4.7 Quote
Claude Opus 4.8 scores 55.8%, a meaningful improvement over Claude Opus 4.7 at 51.9% and Claude Sonnet 4.6 at 41.7%.
51.9% p. 228, section 8.14.1 HealthBench Professional; Figure 8.14.A p. 229 Source of record HealthBench Professional Claude Sonnet 4.6 Quote
Claude Opus 4.8 scores 55.8%, a meaningful improvement over Claude Opus 4.7 at 51.9% and Claude Sonnet 4.6 at 41.7%.
41.7% p. 228, section 8.14.1 Prints a different number GPT-5.5 Instant System CardOpenAI, first-party, 5 May 20261 result
Open the document deploymentsafety.openai.com, retrieved 7 Sep 2026
Board Row Printed Where Role HealthBench Professional GPT-5.5 Instant Quote
HealthBench Professional 37.6 (40.4, 2,973) 35.7 (38.3, 2,872) 32.9 (33.8, 2,285) 38.4 (40.7, 2,775)
38.4 Section 4.1 HealthBench, Table 5 (reported as length-adjusted score (unadjusted, mean response length in characters)), column GPT-5.5 INSTANT Source of record