AI Chatbots in Healthcare: Outperforming Doctors in Diagnosis but Requiring Safeguards Against Overprescribing

Artificial intelligence (AI) chatbots are increasingly becoming a part of the healthcare landscape, often surpassing human doctors in diagnostic accuracy. These tools, powered by advanced large language models like ChatGPT, ERNIE Bot, and DeepSeek, are being tested and implemented across hospitals, clinics, and even personal devices. They are designed to assist with clinical decision-making, especially where medical staffing is limited, by engaging in symptom assessment and suggesting possible diagnoses.

Recent research published in npj Digital Medicine highlights both the potential and the pitfalls of AI in health care. The study conducted simulated patient interactions to compare AI chatbot performance with that of human primary care physicians. It found that while chatbots achieved high diagnostic accuracy, they also tended to recommend unnecessary tests and medications at alarmingly high rates—over 90% and 50%, respectively. For instance, in cases of asthma symptoms, some AI systems suggested antibiotics or expensive scans, which are not supported by clinical guidelines.

Moreover, the research uncovered disparities in AI recommendations based on patient characteristics like age, income, or health status. Older and wealthier patients often received more tests and prescriptions, raising concerns about inequality and overuse. These findings underscore the importance of implementing safeguards such as equity checks, audit trails, and human oversight — particularly for high-stakes decisions — before deploying AI tools widely.

AI’s role in healthcare is promising, especially in regions with limited access to medical professionals, but it also presents risks: overprescribing, increased costs, and reinforcement of existing inequalities. As AI continues to evolve, the emphasis must be on responsible development and deployment that prioritizes safety, fairness, and transparency.

The call to action from these findings is clear: co-design AI systems with safety and justice in mind. By understanding both their capabilities and limitations, policymakers, developers, and health providers can foster AI solutions that enhance healthcare access while safeguarding against harm. This ongoing research aims to guide the responsible integration of AI into healthcare systems globally, ensuring equitable benefits for all communities.