Risks of AI Mental Health Tools Highlighted in New Study

A recent comprehensive study raises concerns about the safety and efficacy of artificial intelligence (AI) tools used in mental health care. Although AI-based chatbots have been promoted as accessible, low-cost solutions for mental health support, new research from Stanford University emphasizes significant risks associated with their deployment. The study, which will be presented at the ACM Conference on Fairness, Accountability, and Transparency and published on the arXiv preprint server, reveals that many popular therapy chatbots exhibit biases and responses that could jeopardize patient safety.

Therapy remains a vital component of mental health treatment, yet nearly 50% of individuals who could benefit from such services are unable to access them. AI chatbots are seen as a potential bridge to meet this demand, but current models may inadvertently reinforce stigma or provide inappropriate guidance. The research team evaluated five prominent AI therapy platforms, including 'Pi' and 'Noni' from therapy platform 7cups, and 'Therapist' from Character.ai, to assess their adherence to established therapeutic guidelines.

The experiments uncovered troubling findings: many chatbots demonstrated increased stigma toward conditions like schizophrenia and alcohol dependence, which could discourage patients from seeking help. Furthermore, when responding to symptoms such as suicidal thoughts or delusions, some AI tools failed to recognize warning signs and even suggested dangerous solutions, such as providing information about bridges in the context of suicidal ideation. This highlights the dangers of relying on AI systems in sensitive mental health situations.

Experts warn that these risks stem from fundamental differences between human and AI therapists. Human therapists are trained to show empathy, avoid stigmatization, and handle crises effectively. AI models, however, often lack the nuanced understanding required for safety-critical applications, raising questions about their current suitability for direct therapy roles.

Despite these concerns, researchers suggest that AI can still serve a supportive role in mental health care when integrated carefully. Proposed applications include assisting therapists with administrative tasks, training scenarios, or providing non-critical support such as journaling or reflection exercises. Ultimately, the study underscores the importance of critical evaluation and regulation of AI tools in mental health, emphasizing that their role should be carefully defined to avoid potential harm.

For more details, see the original study by Jared Moore et al. (2025) at arXiv.