Quick Facts

• AI chatbots correctly diagnosed medical conditions only 35% of the time when tested with 1,300 real users, down from 95% in lab tests
• More than 40 million people daily turn to ChatGPT for health information, with 230 million weekly users asking health questions globally
• Patient safety organization ECRI named AI chatbots in healthcare the top health technology hazard for 2026

AI chatbots are failing to provide accurate medical advice to real users, despite millions of people turning to these tools daily for health guidance.

A new study published in Nature Medicine found that when 1,300 UK participants presented medical scenarios to AI chatbots conversationally, the systems correctly diagnosed specific conditions only 35% of the time. The chatbots recommended appropriate actions just 44% of the time.

This represents a dramatic drop from laboratory performance, where the same AI systems achieved 95% accuracy in identifying medical issues and recommended appropriate actions 56% of the time in controlled tests.

“These findings highlight the difficulty of building AI systems that can genuinely support people in sensitive, high-stakes areas like health,” said Dr. Rebecca Payne, lead medical practitioner on the study from Oxford University. “Despite all the hype, AI just isn’t ready to take on the role of the physician.”

The research comes as more than 40 million people daily turn to ChatGPT for health information, according to OpenAI analysis. Over 230 million people globally ask health-related questions on ChatGPT weekly.

ECRI, an independent patient safety organization, placed AI chatbots in healthcare at the top of its 2026 list of most significant health technology hazards. The organization documented cases where chatbots suggested incorrect diagnoses, recommended unnecessary testing, and even invented body parts while sounding authoritative.

“One chatbot gave dangerous advice when ECRI asked whether it would be acceptable to place an electrosurgical return electrode over the patient’s shoulder blade,” said Marcus Schabacker, ECRI’s president and CEO. “The chatbot incorrectly stated that placement was appropriate – advice that, if followed, would leave the patient at risk of burns.”

The timing is significant as OpenAI launched ChatGPT Health in January 2026, allowing users to connect medical records and wellness apps to the chatbot. The company emphasizes the tool is not intended for diagnosis or treatment.

Researchers identified key technical limitations behind the poor performance. AI systems predict word sequences based on training data patterns rather than truly understanding medical context. They are programmed to always provide confident-sounding answers, even when unreliable.

“AI possesses the medical knowledge, but people struggle to extract useful advice from it,” said Adam Mahdi, Oxford mathematician leading the study.

The study used 10 expert-designed medical scenarios to test real-world performance. Roughly 66% of physicians already use AI in their practice, according to American Medical Association data from 2025.

Privacy concerns add another layer of risk. Neither OpenAI nor other major chatbot providers are bound by healthcare privacy laws like HIPAA, meaning health data shared with chatbots lacks the same legal protections as information shared with doctors.

Read more: As more people turn to chatbots for health advice, studies say they may be led astray