AI Diagnostics: Microsoft claims superior accuracy over generalist doctors in complex cases

For patients in New England and beyond, the landscape of medical diagnostics may shift as artificial intelligence begins to rival human expertise in difficult clinical situations. Microsoft has released findings indicating that its new AI system, MAI-DxO, correctly identified diagnoses in complex simulated cases at a rate more than double that of generalist physicians. While this does not yet mean robots are replacing doctors, it signals a potential change in how challenging medical puzzles are solved in the coming years.

The core of the advancement is a system that functions less like a single tool and more like a virtual medical team. Rather than relying on a single prompt to an AI model, MAI-DxO coordinates multiple AI agents that work together. These agents simulate a panel of specialists, engaging in a process where they question a patient, order necessary tests, and debate potential diagnoses among themselves before arriving at a conclusion. This collaborative approach is designed to mimic the way human specialist teams tackle ambiguous medical issues.

In a rigorous simulation using difficult cases from the New England Journal of Medicine, the AI system demonstrated a high level of proficiency. It arrived at the correct diagnosis 83% of the time. By comparison, primary care doctors and general internists, who were working individually with standard reference materials, were correct only 36% of the time. It is important to note that these specific cases are typically handled by specialists or teams in real life, so the comparison highlights a gap between generalist capabilities and the AI’s specialized reasoning rather than a direct head-to-head with a full specialist team.

The performance of the system is heavily dependent on the underlying AI model it uses. Microsoft tested the orchestration framework with models from OpenAI, Google, Anthropic, Meta, and DeepSeek. The highest accuracy scores were achieved when paired with OpenAI’s GPT-5 and GPT-5.5. The data suggests that much of the improvement in diagnostic accuracy comes from the power of these newer, larger models, with the multi-agent coordination providing an additional layer of efficiency and precision on top of that foundation.

Cost and speed are critical factors for widespread adoption, and the study measured these metrics alongside accuracy. When using the GPT-5 model alone with a basic prompt, the system cost approximately $8,400 per case and achieved a 78.9% accuracy rate on complex issues. The MAI-DxO orchestration reduced this cost by about $3,000 while raising accuracy by roughly four percentage points. However, in emergency department scenarios, the improvement in both cost and accuracy was minimal when comparing the orchestrated system to the standalone model.

The benefits of the orchestration layer appear more pronounced when working with less powerful AI models. For instance, Google’s Gemini 2.5 Flash model scored 46.4% on complex cases on its own, but its performance jumped to nearly 69% when guided by the MAI-DxO framework. Despite this significant boost, the gap between weaker and stronger base models remained larger than the enhancement provided by the coordination system, suggesting that raw model capability remains the primary driver of high-level diagnostic performance.

Microsoft is moving beyond simulations to test these capabilities in real-world settings. Through a new partnership with Beth Israel Deaconess Medical Center in Boston, the company will evaluate the clinical reasoning of the system across various hospital environments. Patients will continue to receive standard care, including triage and specialist consultations, while those who consent will also interact with the AI. This allows researchers to compare the AI’s recommendations directly against the decisions made by human doctors in a live clinical environment.

Mustafa Suleyman, CEO of Microsoft AI, views this work as a step toward "medical superintelligence," defined as the ability to surpass human-led systems in medical reasoning. He stated that the company is collaborating with the Mayo Clinic to train a dedicated foundation model for healthcare, with a release planned for next year. Suleyman predicts that by 2027, the technology will enable a breakthrough in how medical records are processed, allowing for highly accurate predictions about how a patient's condition might unfold, which he describes as "gold dust" for clinicians.

The release of these results places Microsoft in direct competition with other tech giants like Google and OpenAI, which have also claimed that their AI systems can match or outperform doctors. Microsoft’s strategy focuses not just on benchmarking individual tasks, but on integrating with existing healthcare systems to improve overall outcomes. As generative AI in healthcare is still in its early stages, industry leaders expect capabilities to improve significantly within the next two to three years, driven by continuous iteration and real-world usage data.

As the technology advances, the goal is to move beyond simple diagnostic assistance to comprehensive decision support. Microsoft aims to build state-of-the-art clinical tools that expand its current offerings, which include dictation and radiology software. The company believes that by creating a flywheel of improvement through real-world deployment, it can accelerate the introduction of these tools, potentially transforming how medical care is delivered in the near future.