Clinical Benchmarks

HealthAgentBench

Agent harnesses completing realistic terminal-based healthcare tasks built from real clinical artifacts.

Microsoft Research12 rows10 of 12 models on the boardmeasured to Jul 2026

Official pagePaper
More about this boardLess

Agentic task success in realistic terminal-based healthcare environments built from real clinical artifacts; evaluates agent harnesses (Claude Code, Codex, Copilot) end to end, not bare models.

Paper: arXiv 2606.31179. The rows are agent harnesses rather than bare models, and the board is run by Microsoft Research; Copilot, Microsoft's own harness, does not top it.

Published by Microsoft Research, released Jul 2026. 54 tasks across 7 environments; 162 trials (3 attempts per task). Mean task success rate, 0-100%, higher better; cost per task also reported.

Headline metric
pooled task success rate across 3 attempts x 54 tasks (162 trials)
Rows on the board
12
Board last updated
27 Jul 2026

Ranking

  1. 1Claude Code (Opus 5)Anthropic · $3.3/task55
    Printed as 55%Official leaderboard, measured Jul 2026Configuration: $3.3/task; harness+model evaluated jointly
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 1, Success Rate column
    1 Claude Code (Opus 5) 55% $3.3
    Every result from this document

    Link to this row

  2. 2Codex (GPT-5.6-sol)OpenAI · $5.2/task45
    Printed as 45%Official leaderboard, measured Jul 2026Configuration: $5.2/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 2, Success Rate column
    2 Codex (GPT-5.6-sol) 45% $5.2
    Every result from this document

    Link to this row

  3. 3Codex (GPT 5.5)OpenAI · $2.8/task42
    Printed as 42%Official leaderboard, measured Jul 2026Configuration: $2.8/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 3, Success Rate column
    3 Codex (GPT 5.5) 42% $2.8
    Every result from this document

    Link to this row

  4. 4Copilot (Opus 4.8)Microsoft/Anthropic · $3.1/task36
    Printed as 36%Official leaderboard, measured Jul 2026Configuration: $3.1/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 4, Success Rate column
    4 Copilot (Opus 4.8) 36% $3.1
    Every result from this document

    Link to this row

  5. 5Copilot (GPT 5.5)Microsoft/OpenAI · $2.6/task35
    Printed as 35%Official leaderboard, measured Jul 2026Configuration: $2.6/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 5, Success Rate column
    5 Copilot (GPT 5.5) 35% $2.6
    Every result from this document

    Link to this row

  6. 6Claude Code (Opus 4.8)Anthropic · $4.0/task32
    Printed as 32%Official leaderboard, measured Jul 2026Configuration: $4.0/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 6, Success Rate column
    6 Claude Code (Opus 4.8) 32% $4.0
    Every result from this document

    Link to this row

  7. 7Codex (GPT 5.4)OpenAI · $1.3/task28
    Printed as 28%Official leaderboard, measured Jul 2026Configuration: $1.3/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 7, Success Rate column
    7 Codex (GPT 5.4) 28% $1.3
    Every result from this document

    Link to this row

  8. 8Claude Code (Opus 4.7)Anthropic · $4.8/task27
    Printed as 27%Official leaderboard, measured Jul 2026Configuration: $4.8/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 8, Success Rate column
    8 Claude Code (Opus 4.7) 27% $4.8
    Every result from this document

    Link to this row

  9. 9Codex (GPT 5.3)OpenAI · $1.0/task22
    Printed as 22%Official leaderboard, measured Jul 2026Configuration: $1.0/task; Codex harness; mean success across 3 attempts × 54 tasks
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Homepage leaderboard, rank 9, Success Rate and cost/task; displayed 2026-09-30.
    Agent | Success Rate | cost/task Codex (GPT 5.3) 22% $1.0
    Every result from this document

    Link to this row

  10. 10Claude Code (Opus 4.6)Anthropic · $4.1/task19
    Printed as 19%Official leaderboard, measured Jul 2026Configuration: $4.1/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 10, Success Rate column
    10 Claude Code (Opus 4.6) 19% $4.1
    Every result from this document

    Link to this row

All 12 rowsTop 10 only
  1. 11Claude Code (Sonnet 4.6)Anthropic · $2.9/task17
    Printed as 17%Official leaderboard, measured Jul 2026Configuration: $2.9/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 11, Success Rate column
    11 Claude Code (Sonnet 4.6) 17% $2.9
    Every result from this document

    Link to this row

  2. 12Codex (GPT 5.4 Mini)OpenAI · $0.6/task16
    Printed as 16%Official leaderboard, measured Jul 2026Configuration: $0.6/task
    HealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 12, Success Rate column
    12 Codex (GPT 5.4 Mini) 16% $0.6
    Every result from this document

    Link to this row

Mean task success rate, 0-100%, higher better; cost per task also reported. Bar colour: the lab; a pale bar with an outline is a number the model’s own lab reported. Select a row for its quote, page and document.

Documents

3 documents support this board