Research Shows Human Medical Staff Outperform AI in Emergency Patient Triage

Recent studies presented at the European Emergency Medicine Congress highlight that physicians and nurses excel over artificial intelligence (AI) when it comes to triaging patients in emergency departments. The research involved comparing the accuracy and efficiency of clinical staff with an AI model, ChatGPT (version 3.5), in classifying the urgency of medical cases based on reports from the PubMed database.

Dr. Renata Jukneviciene, a postdoctoral researcher from Vilnius University, led the study to address ongoing issues with overcrowding and staff workload in emergency settings. The team distributed questionnaires to six doctors and 51 nurses at Vilnius University Hospital Santaros Klinikos, asking them to triage randomly selected cases. These same cases were evaluated by ChatGPT for comparison.

The findings revealed that AI generally underperformed relative to human clinicians, achieving an overall accuracy of 50.4%, whereas nurses scored 65.5%, and doctors 70.6%. Sensitivity, which measures the AI’s ability to identify genuinely urgent cases, was notably lower at 58.3%, compared to 73.8% for nurses and 83.0% for doctors.

Interestingly, AI showed higher performance in identifying the most critical cases—those requiring immediate intervention—outperforming nurses in accuracy (27.3% vs. 9.3%) and specificity (27.8% vs. 8.3%). This suggests that AI tends to over-triage, erring on the side of caution by flagging more patients as critical, which could both protect patient safety and lead to unnecessary resource use.

In terms of surgical and therapeutic cases, human clinicians still outshined AI. Doctors scored 68.4% reliability in surgical case triage, while nurses scored 63%. AI lagged behind at 39.5%. Conversely, in cases involving medications or non-invasive therapies, AI performed slightly better than nurses.

Dr. Jukneviciene emphasized that AI should not replace trained medical staff but could serve as a supplementary tool—particularly useful in supporting less experienced staff and improving prioritization during critical situations. She cautioned that over-reliance on AI might cause inefficiencies and stressed the importance of human oversight.

The researchers plan further studies with more advanced AI models, larger participant groups, and additional features like ECG interpretation to enhance AI’s role in emergency medicine. Limitations noted include the small sample size, single-center setting, and the offline nature of AI analysis, which did not allow real-time application or interaction with patient vital signs.

Experts agree that while AI holds promise for areas like imaging and support, it cannot fully replace the nuanced judgment of medical professionals. Proper integration and ongoing testing are essential to harness AI’s benefits without compromising patient safety.