Scientists Caution Against Relying on AI for Stroke Care

Recent research emphasizes the current limitations of artificial intelligence (AI) models in supporting stroke care, including prevention, diagnosis, treatment, and recovery. Although advanced chatbots like ChatGPT-4o, Claude 3 Sonnet, and Gemini Ultra 1.0 have shown promise in providing health information, they often fall short of the clinical accuracy and safety required for medical decision-making. An international study involving researchers from National Taiwan University and Harvard T.H. Chan School of Public Health evaluated these generative AI models through realistic clinical scenarios and multiple prompt strategies. Results revealed that, on average, the AI models scored between 48 and 56 out of 100, below the threshold of 60 deemed necessary for safe clinical use. While some gains in empathy and clarity were observed with specific prompting techniques, none of the models consistently delivered reliable advice across all stroke care stages, especially in high-stakes areas like acute treatment. Experts warn that despite their impressive capabilities in disseminating general health information, AI chatbots currently require significant human oversight to avoid potential risks. The study underscores the importance of cautious integration of AI in healthcare, advocating that these tools should complement, not replace, clinician expertise. Deploying AI responsibly involves rigorous validation, clear governance, and maintaining the essential role of healthcare professionals to ensure patient safety and effective care.