Baichuan M4 Tops HealthBench: Medical Large Models Reach New Heights, GPT-5.5 Left Behind by 10 Points

On June 22, Baichuan Intelligence, in collaboration with Tsinghua University, officially released the new generation Baichuan-M4 medical-enhanced large model. The release was unsurprising—six months ago the M3 had already soundly beaten GPT-5.2, and M4 simply reaffirms Baichuan’s dominance in the medical vertical. But the interesting details lie in the numbers: HealthBench composite score of 68.6, ahead of GPT-5.5 by more than 10 points; Hard subset lead of 15.9 points; hallucination rate of 3.3%. This isn’t just “surpassing,” it’s a generational gap.

One-Sentence Conclusion

M4 took first place worldwide on OpenAI’s own HealthBench, HealthBench Hard, and HealthBench Professional rankings—all at once—beating GPT-5.5, Claude Opus 4.7, and DeepSeek-V4-Pro. Especially noteworthy is the Hard subset—this leaderboard is designed to test complex clinical decision-making, essentially “hell difficulty” meant to frustrate general models. In the past six months, only a handful of models have scored above 50 here. M4 leads the second-place model by 15.9 points in Hard, widening the gap since the M3 era.

Looking back at the M series’ release cadence: August last year saw M2 (32B, open source), January this year M3 (235B, open source), and now M4 in June. Each generation has launched right on the heels of OpenAI’s new models, and each has pushed the hallucination rate lower: Above 4% with M2, down to 3.5% with M3, and now 3.3% with M4. In medicine, where a single wrong character can be life-threatening, 0.2 percentage points is no small number.

What Exactly Changed in M4

Judging by the rankings alone, M4 looks like a routine upgrade from M3. But dissecting the technical report reveals that this time the changes are actually redefining what a “medical large model” should look like. Baichuan CEO Wang Xiaochuan, at an academic forum in Tsinghua last May, outlined three rigid requirements for medical scenarios: low hallucination, strong evidence-based reasoning, and active questioning—and general large models meet none of them. M4’s upgrades are precisely targeted to meet these three requirements.

First Change: Actively Asking Follow-Up Questions, Not Rushing to Give Answers

This builds upon and deepens the “serious consultation paradigm” from the M3 era. General large models have a bad habit—answering whatever is asked, reasoning only with the information the user gives. This leads to situations where the user says “I have chest pain,” and the model immediately lists ten possible diseases. It looks comprehensive, but it’s clinically useless.

M4’s logic is the opposite: first detect any signs of critical conditions, then ask about the nature of the symptoms, underlying causes, accompanying signs, and medical history. It prioritizes exclusion over inference. This subtle difference shows up clearly in SCAN-bench evaluation results.

SCAN-bench was built jointly by Baichuan and over 150 frontline doctors, inspired by OSCE (Objective Structured Clinical Examination) used in medical education for decades—testing not how much textbook material you’ve memorized, but whether you can run through diagnostic processes like a doctor. The core is dynamic multi-round dialogue: evaluators role-play patients, revealing information gradually based on the model’s questions, and the model must decide what to ask and how far to probe before diagnosing.

Evaluation results:

• M4 scored 79.0 in initial consultation, 74.7 in follow-up
• Clearly ahead of GPT-5.5, DeepSeek-V4-Pro, Claude Opus 4.7
• Scored above human doctor baseline in all four SCAN dimensions

That last point is key. In the M3 era, Baichuan claimed its models surpassed human baseline performance, but skeptics argued the evaluation scope was too narrow. M4’s results span both initial and follow-up consultations, exceeding baseline across the board, making the claim harder to dispute.

Second Change: Full Care Path Memory

This is arguably the most underestimated part of the upgrade.

The problem with general models: Once a session ends, they forget who you are. For chronic disease patients like diabetics, who check HbA1c every 3 months, long-term management is essentially impossible for general models. You must paste your medical history, medication record, lab trends into the prompt every time—poor experience and prone to errors.

M4 integrates historical medical records, multi-round consultation logs, lab trends, and medication feedback into one “full care path memory.” The model knows your past diagnoses, how various metrics have changed, and whether previously prescribed medications have side effects.

In long-context clinical memory evaluation:

• M4 scored 86.9, highest in its class
• Up 21.1 points from M3

A 21-point improvement essentially jumps from “barely usable” to “clinically deployable.” Paired with Baichuan’s own AI family doctor “Baixiaoyi,” this could support complete long-term health management.

Third Change: Evidence Anchoring + 1000+ Clinical Pathway Units

Another stubborn challenge for medical models: “Where does the evidence come from?” Early solutions used RAG—attach a literature database, have the model generate output, then tag “Reference [3].” But often, the cited references don’t match the conclusions; essentially, the model is still guessing.

M4’s “evidence anchoring” tightens this requirement: every single medical conclusion generated must precisely correspond to a specific paragraph in the original paper or guideline—not vague citations, but paragraph-level location. Combined with a “six-source evidence-based paradigm,” the model’s search scope is restricted to authoritative medical sources, excluding open web data.

Going further, M4 decomposed authoritative guidelines, expert consensus, and real diagnostic processes into over 1000 standardized, reusable clinical pathway units, covering over 200 diseases, each defined and verified by seasoned clinical experts. This sounds straightforward but the workload is massive—breaking down a single guideline into atomic actionable units requires repeated alignment between medical experts and engineering teams.

In Baichuan’s own evidence-based medicine evaluation, Baichuan-EBM:

• M4 evidence citation accuracy: 90.0
• GPT-5.5: 54.7
• OpenEvidence (specialized in evidence-based medicine): 55.9

Note that OpenEvidence is a specialist in the evidence-based medical field, yet M4 beats it by 35 points. This gap shows that in certain contexts, vertical-domain engineering know-how can indeed surpass general large models’ parameter scale.

How to View General Models vs. Medical-Specific Models

In his May speech, Wang Xiaochuan cited figures: general models’ medical QA responses are judged “problematic” ~50% of the time, with differential diagnosis error rates over 80%, and accuracy for real patient self-service drops from 94.9% to 34.5%.

That last number is the core issue. In academic evaluations, general models appear competent because the test cases are structured—complete medical history, clear problem statements, even diagnosis options provided. But real patients don’t talk like that. They say “I haven’t felt right these past two days,” “I can’t sleep well at night,” “That medicine seemed to cause a reaction.” General models can’t handle such unstructured input, so an accuracy drop is unsurprising.

M4’s design philosophy is the opposite: assume the user can’t describe, can’t ask, and doesn’t know what information matters. The model must actively take control of the consultation process. This approach is actually suboptimal in general conversation—you don’t want GPT to ask you ten questions before writing code—but in medical scenarios, it’s essential.

This answers the controversy: Do we need dedicated medical models? The past two years have seen heated debate; OpenAI and Anthropic are clearly increasing their focus—OpenAI launched ChatGPT Health, Anthropic introduced Claude for Healthcare. Their strategy is “general model + vertical data,” without dedicated medical base training. M4’s results at least prove that in medical contexts with rigid demands for low hallucination, evidence-based reasoning, and consultation workflow, a specially trained medium-scale model can outperform general large models—and may cost less.

Some Points Not Fully Addressed

Worth watching:

1. Will M4 be open-sourced? M2 and M3 were both open-sourced; no clear Hugging Face or GitHub repository info yet for M4. Baichuan’s strategy has been to release weights shortly after launch, so M4 likely will be, possibly larger than M3.
2. Speed of clinical deployment: Medical AI has been stalled by regulation for years. M4’s capabilities are ready, but integrating into hospital systems and obtaining device certifications requires a long compliance process.
3. Product form of Baixiaoyi: Released in May alongside M4, “Baixiaoyi” AI family doctor app is M4’s main consumer-facing outlet. Whether evaluation scores translate into real user retention and willingness to pay is another question.
4. Will SCAN-bench be widely adopted? Baichuan’s self-built evaluation system has faced “player and referee” criticism. HealthBench, created by OpenAI, is relatively neutral, making M4’s top spot more convincing. SCAN-bench results will gain weight if other vendors also adopt it.

Final Thoughts

M4’s release may not be surprising, but it extends the capability boundaries of medical large models. HealthBench composite score lead of 10 points, Hard subset lead of 15.9 points, evidence accuracy lead of 35 points—these are gaps too large to be explained by statistical noise.

Since M2, Baichuan has focused relentlessly on medical AI, now reaching the fourth generation with M4—a rare example of sustained vertical-domain dedication in China. Over the past two years, many companies in AI healthcare have failed; few remain, let alone keep pushing technological breakthroughs. M4 at least proves this: in a sufficiently narrow and deep domain, thorough work delivers value, and is clearer-headed than chasing general model tail lights.

References

• IT Home: Baichuan releases new generation medical-enhanced large model M4, tops OpenAI medical evaluation - Coverage of M4’s key results and technical details
• GitHub: Baichuan-M2-32B repository - Open-source repository of M2, reference for technical architecture
• Zhihu: Baichuan 32B medical model tops global rankings - M2 era comparison between Baichuan medical model and OpenAI