Clinical Benchmarks
Menu

First, Do NOHARM (v2)

How often, and how severely, model consultation recommendations contain potentially harmful errors.

Stanford/Harvard consortium17 rows17 of 19 models on the boardmeasured to Sep 2026

Official pagePaper
More about this boardLess

Frequency and severity of potentially harmful errors in LLM-generated medical consultation recommendations (Numerous Options Harm Assessment for Risk in Medicine); primary-care-to-specialist consults.

Paper: arXiv 2512.01241. The v1 study found potential for severe harm in up to 24.6 percent of directly applied recommendations, with errors of omission behind more than 80 percent of the severe cases.

Published by Stanford/Harvard-led consortium (50+ researchers incl. 29 board-certified physicians); hosted by ARISE, released Dec 2025. 1,100 consultation cases, 10 specialties, 12,747 expert annotations on 4,249 management options. Percentage safety score, higher better.

Headline metric
F1 (Weighted)
Board marked as preview
Yes
Board last updated
15 Aug 2026
Models in the full dataset
43
Models in the board's default view
19

Ranking

  1. 1LiSA 2.5AMBOSS · First Do NOHARM v2 overall86.2
    Printed as 86.2%Official leaderboard, measured Sep 2026Configuration: First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
    ARISE MAST technical leaderboard official leaderboard, ARISE. First Do NOHARM v2, overall leaderboard; LiSA 2.5 row; displayed 2026-09-30.
    First Do NOHARM v2 overall metric across 19 models 1LiSA 2.5RAGAMBOSS 86.2%
    Every result from this document

    Link to this row

  2. 2Doximity Ask 6.1Doximity · First Do NOHARM v2 overall84.5
    Printed as 84.5%Official leaderboard, measured Sep 2026Configuration: First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
    ARISE MAST technical leaderboard official leaderboard, ARISE. First Do NOHARM v2, overall leaderboard; Doximity Ask 6.1 row; displayed 2026-09-30.
    First Do NOHARM v2 overall metric across 19 models 2Doximity Ask 6.1RAGDoximity 84.5%
    Every result from this document

    Link to this row

  3. 3OpenEvidenceOpenEvidence · First Do NOHARM v2 overall80.0
    Printed as 80.0%Official leaderboard, measured Sep 2026Configuration: First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
    ARISE MAST technical leaderboard official leaderboard, ARISE. First Do NOHARM v2, overall leaderboard; OpenEvidence row; displayed 2026-09-30.
    First Do NOHARM v2 overall metric across 19 models 3OpenEvidenceRAGOpenEvidence 80.0%
    Every result from this document

    Link to this row

  4. 4Glass 5.6 MaxGlass Health · First Do NOHARM v2 overall79.7
    Printed as 79.7%Official leaderboard, measured Sep 2026Configuration: First Do NOHARM v2 overall; preview; RAG clinical product; run date unpublished
    ARISE MAST technical leaderboard official leaderboard, ARISE. First Do NOHARM v2, overall leaderboard; Glass 5.6 Max row; displayed 2026-09-30.
    First Do NOHARM v2 overall metric across 19 models 4Glass 5.6 MaxRAGGlass Health 79.7%
    Every result from this document

    Link to this row

  5. 4Muse Spark 1.1Meta79.7
    Printed as 79.7%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
    5Muse Spark 1.1Meta 79.7%
    Every result from this document

    Link to this row

  6. 6Claude Opus 5Anthropic · v2 run on ARISE74.6
    Printed as 74.6%Official leaderboard, measured Aug 2026Configuration: v2 run on ARISE; 19 models on the board
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
    6Claude Opus 5Anthropic 74.6%
    Every result from this document

    Link to this row

  7. 7Kimi K3Moonshot AI74.0
    Printed as 74.0%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
    7Kimi K3OSSMoonshot AI 74.0%
    Every result from this document

    Link to this row

  8. 8GPT-5.6 SolOpenAI70.1
    Printed as 70.1%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
    8GPT-5.6 SolOpenAI 70.1%
    Every result from this document

    Link to this row

  9. 9GPT-5.5OpenAI70.0
    Printed as 70.0%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
    9GPT-5.5OpenAI 70.0%
    Every result from this document

    Link to this row

  10. 10GPT-5OpenAI · from the Model Leaderboard…68.6
    Printed as 68.6%Official leaderboard, measured Aug 2026Configuration: from the Model Leaderboard SAFETY column (NOHARM v2 F1 weighted, shown with CI); not in the Latest Flagships ranking
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, Model Leaderboard (Top 10 shown), row 10, SAFETY column
    10 GPT-5 72.6%±2.3 68.6%±4.5 80.3%±4.4 30.1%±9.0 44.2%±2.5 45.4%±1.3 73.1%±3.9 73.7%±2.7 46.6%±0.0
    Every result from this document

    Link to this row

All 17 rowsTop 10 only
  1. 11Claude Fable 5Anthropic65.0
    Printed as 65.0%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
    10Claude Fable 5Anthropic 65.0%
    Every result from this document

    Link to this row

  2. 12Gemini 3.1 ProGoogle62.6
    Printed as 62.6%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view), plus Model Leaderboard SAFETY column
    11Gemini 3.1 ProGoogle 62.6%
    Every result from this document

    Link to this row

  3. 13Gemini 2.5 ProGoogle61.9
    Printed as 61.9%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
    12Gemini 2.5 ProGoogle 61.9%
    Every result from this document

    Link to this row

  4. 14Qwen3.5 397B A17BAlibaba61.1
    Printed as 61.1%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
    13Qwen3.5 397B A17BOSSAlibaba 61.1%
    Every result from this document

    Link to this row

  5. 15Kimi K2.6Moonshot AI59.1
    Printed as 59.1%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
    14Kimi K2.6OSSMoonshot AI 59.1%
    Every result from this document

    Link to this row

  6. 16GLM 5.1Zhipu · First Do NOHARM v2 overall57.9
    Printed as 57.9%Official leaderboard, measured Sep 2026Configuration: First Do NOHARM v2 overall; preview; open-weight model; run date unpublished
    ARISE MAST technical leaderboard official leaderboard, ARISE. First Do NOHARM v2, overall leaderboard; GLM 5.1 row; displayed 2026-09-30.
    First Do NOHARM v2 overall metric across 19 models 15GLM 5.1OSSZ.ai 57.9%
    Every result from this document

    Link to this row

  7. 17DeepSeek R1DeepSeek55.8
    Printed as 55.8%Official leaderboard, measured Aug 2026
    MAST technical leaderboard (First Do NOHARM v2 and per-benchmark results) official leaderboard, ARISE AI Research Network, 15 Aug 2026. arise-ai.org/mast/technical, 'First Do NOHARM v2 overall metric across 19 models' ranking (Latest Flagships view)
    16DeepSeek R1OSSDeepSeek 55.8%
    Every result from this document

    Link to this row

Percentage safety score, higher better. Bar colour: the lab; a pale bar with an outline is a number the model’s own lab reported. Select a row for its quote, page and document.

Documents

2 documents support this board