Waldo: Advanced AI Tool Detects Hidden Health Risks on Social Media

A groundbreaking artificial intelligence tool named Waldo has been developed to analyze social media platforms, aiming to identify adverse health events associated with various consumer products. Published in the open-access journal PLOS Digital Health, this study led by John Ayers from the University of California, San Diego, highlights Waldo's ability to enhance post-market surveillance efforts. Today, monitoring the safety of consumer health products, such as dietary supplements and cannabis-derived items, is crucial for public health. However, traditional adverse-event reporting relies heavily on voluntary submissions from healthcare professionals and manufacturers, which can result in underreporting.

In response to the surge in new consumer health products, researchers tested Waldo's effectiveness in scanning social media, particularly Reddit posts, to detect descriptions of adverse events. When compared to human annotations, Waldo demonstrated an accuracy rate of 99.7%, outperforming general-purpose chatbots like ChatGPT on identical tasks. Within a dataset of over 437,000 Reddit posts, Waldo identified nearly 29,000 potential harm reports, of which 86% were confirmed as true adverse events upon manual review.

The team has made Waldo openly accessible so that researchers, clinicians, and regulators worldwide can leverage this technology. According to the researchers, Waldo's superior performance signifies a major advancement in social media monitoring for health risks, with applications extending beyond cannabis products to other unregulated consumer health items.

Lead author Karan Desai emphasizes that online health experiences are valuable safety signals, and capturing these helps reveal harms often missed by traditional systems. The project demonstrates how digital health tools like Waldo can revolutionize post-market safety surveillance, fostering open science and protecting patient well-being.

By democratizing access to such tools, the researchers aim to accelerate the identification of health risks and support regulatory decisions, ultimately improving public health outcomes.