HealthAgentBench
Agent harnesses completing realistic terminal-based healthcare tasks built from real clinical artifacts.
Microsoft Research12 rows10 of 12 models on the boardmeasured to Jul 2026
More about this boardLess
Agentic task success in realistic terminal-based healthcare environments built from real clinical artifacts; evaluates agent harnesses (Claude Code, Codex, Copilot) end to end, not bare models.
Paper: arXiv 2606.31179. The rows are agent harnesses rather than bare models, and the board is run by Microsoft Research; Copilot, Microsoft's own harness, does not top it.
Published by Microsoft Research, released Jul 2026. 54 tasks across 7 environments; 162 trials (3 attempts per task). Mean task success rate, 0-100%, higher better; cost per task also reported.
- Headline metric
- pooled task success rate across 3 attempts x 54 tasks (162 trials)
- Rows on the board
- 12
- Board last updated
- 27 Jul 2026
Ranking
- Printed as 55%Official leaderboard, measured Jul 2026Configuration: $3.3/task; harness+model evaluated jointlyHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 1, Success Rate column
1 Claude Code (Opus 5) 55% $3.3
Every result from this document- Also printed in HealthAgentBench detailed results: 55%, Detailed results table, rank 1
- Printed as 45%Official leaderboard, measured Jul 2026Configuration: $5.2/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 2, Success Rate column
2 Codex (GPT-5.6-sol) 45% $5.2
Every result from this document- Also printed in HealthAgentBench detailed results: 45%, Detailed results table, rank 2
- Printed as 42%Official leaderboard, measured Jul 2026Configuration: $2.8/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 3, Success Rate column
3 Codex (GPT 5.5) 42% $2.8
Every result from this document- Also printed in HealthAgentBench detailed results: 42%, Detailed results table, rank 3
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 42%, p. 11, Figure 4 (pooled task success rate, ten agents)
- Printed as 36%Official leaderboard, measured Jul 2026Configuration: $3.1/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 4, Success Rate column
4 Copilot (Opus 4.8) 36% $3.1
Every result from this document- Also printed in HealthAgentBench detailed results: 36%, Detailed results table, rank 4
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 36%, p. 11, Figure 4 (pooled task success rate, ten agents)
- Printed as 35%Official leaderboard, measured Jul 2026Configuration: $2.6/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 5, Success Rate column
5 Copilot (GPT 5.5) 35% $2.6
Every result from this document- Also printed in HealthAgentBench detailed results: 35%, Detailed results table, rank 5
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 35%, p. 11, Figure 4 (pooled task success rate, ten agents)
- Printed as 32%Official leaderboard, measured Jul 2026Configuration: $4.0/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 6, Success Rate column
6 Claude Code (Opus 4.8) 32% $4.0
Every result from this document- Also printed in HealthAgentBench detailed results: 32%, Detailed results table, rank 6
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 32%, p. 11, Figure 4 (pooled task success rate, ten agents)
- Printed as 28%Official leaderboard, measured Jul 2026Configuration: $1.3/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 7, Success Rate column
7 Codex (GPT 5.4) 28% $1.3
Every result from this document- Also printed in HealthAgentBench detailed results: 28%, Detailed results table, rank 7
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 28%, p. 11, Figure 4 (pooled task success rate, ten agents)
- Printed as 27%Official leaderboard, measured Jul 2026Configuration: $4.8/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 8, Success Rate column
8 Claude Code (Opus 4.7) 27% $4.8
Every result from this document- Also printed in HealthAgentBench detailed results: 27%, Detailed results table, rank 8
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 27%, p. 11, Figure 4 (pooled task success rate, ten agents)
- Printed as 22%Official leaderboard, measured Jul 2026Configuration: $1.0/task; Codex harness; mean success across 3 attempts × 54 tasksHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Homepage leaderboard, rank 9, Success Rate and cost/task; displayed 2026-09-30.
Agent | Success Rate | cost/task Codex (GPT 5.3) 22% $1.0
Every result from this document - Printed as 19%Official leaderboard, measured Jul 2026Configuration: $4.1/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 10, Success Rate column
10 Claude Code (Opus 4.6) 19% $4.1
Every result from this document- Also printed in HealthAgentBench detailed results: 19%, Detailed results table, rank 10
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 19%, p. 11, Figure 4 (pooled task success rate, ten agents)
All 12 rowsTop 10 only
- Printed as 17%Official leaderboard, measured Jul 2026Configuration: $2.9/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 11, Success Rate column
11 Claude Code (Sonnet 4.6) 17% $2.9
Every result from this document- Also printed in HealthAgentBench detailed results: 17%, Detailed results table, rank 11
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 17%, p. 11, Figure 4 (pooled task success rate, ten agents)
- Printed as 16%Official leaderboard, measured Jul 2026Configuration: $0.6/taskHealthAgentBench leaderboard official leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026. Leaderboard table (homepage), rank 12, Success Rate column
12 Codex (GPT 5.4 Mini) 16% $0.6
Every result from this document- Also printed in HealthAgentBench detailed results: 16%, Detailed results table, rank 12
- Also printed in HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1): 16%, p. 11, Figure 4 (pooled task success rate, ten agents)
No rows match. Turn a reported-by option back on or clear the chosen models.
Mean task success rate, 0-100%, higher better; cost per task also reported. Bar colour: the lab; a pale bar with an outline is a number the model’s own lab reported. Select a row for its quote, page and document.
| Rank | Model | Lab | Score | Configuration | Reported by | Measured | Source |
|---|---|---|---|---|---|---|---|
| 1 | Claude Code (Opus 5) | Anthropic | 55 | $3.3/task; harness+model evaluated jointly | Official leaderboard | Jul 2026 | Quote and page |
| 2 | Codex (GPT-5.6-sol) | OpenAI | 45 | $5.2/task | Official leaderboard | Jul 2026 | Quote and page |
| 3 | Codex (GPT 5.5) | OpenAI | 42 | $2.8/task | Official leaderboard | Jul 2026 | Quote and page |
| 4 | Copilot (Opus 4.8) | Microsoft/Anthropic | 36 | $3.1/task | Official leaderboard | Jul 2026 | Quote and page |
| 5 | Copilot (GPT 5.5) | Microsoft/OpenAI | 35 | $2.6/task | Official leaderboard | Jul 2026 | Quote and page |
| 6 | Claude Code (Opus 4.8) | Anthropic | 32 | $4.0/task | Official leaderboard | Jul 2026 | Quote and page |
| 7 | Codex (GPT 5.4) | OpenAI | 28 | $1.3/task | Official leaderboard | Jul 2026 | Quote and page |
| 8 | Claude Code (Opus 4.7) | Anthropic | 27 | $4.8/task | Official leaderboard | Jul 2026 | Quote and page |
| 9 | Codex (GPT 5.3) | OpenAI | 22 | $1.0/task; Codex harness; mean success across 3 attempts × 54 tasks | Official leaderboard | Jul 2026 | Quote and page |
| 10 | Claude Code (Opus 4.6) | Anthropic | 19 | $4.1/task | Official leaderboard | Jul 2026 | Quote and page |
| 11 | Claude Code (Sonnet 4.6) | Anthropic | 17 | $2.9/task | Official leaderboard | Jul 2026 | Quote and page |
| 12 | Codex (GPT 5.4 Mini) | OpenAI | 16 | $0.6/task | Official leaderboard | Jul 2026 | Quote and page |
Documents
3 documents support this board
- HealthAgentBench detailed resultsofficial leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026Results it supports
- HealthAgentBench leaderboardofficial leaderboard, Microsoft Research (HealthAgentBench), 27 Jul 2026Results it supports
- HealthAgentBench: A Unified Benchmark Suite of Realistic Agentic Healthcare Environments for Challenging Frontier AI Agents (arXiv 2606.31179v1)paper, Microsoft Research (Liu, Zhang, Qin et al.), 30 Jun 2026Results it supports