AI Detects Online Suicide Content | Korea University Hospital Study

A team led by Dr. Baek Jong-woo at Kyunghee University Hospital has developed an artificial intelligence (AI) system capable of early detection of online content related to suicide, a development announced Wednesday.

The AI system, a collaboration between researchers at Kyunghee University Hospital, Sungkyunkwan University, the Korean Suicide Prevention Association, and Modern York University, analyzes online posts from social media and online communities. The project was initiated to address South Korea’s high suicide rate, the highest among OECD nations, and the challenges of managing online risks in a digital environment, according to Kyunghee University Hospital.

Currently, monitoring for harmful online content is largely conducted by human reviewers. However, this approach faces limitations including the inability to provide 24-hour coverage and the potential for trauma exposure to those reviewing disturbing content. The research team addressed these issues by analyzing 43,244 posts, creating a benchmark dataset vetted by mental health professionals and psychologists to train the AI system.

The AI classifies content into five categories based on risk level: illegal, harmful, potentially harmful, harmless, and non-suicidal. Notably, the system is designed to identify nuanced expressions – including slang, metaphors, and abbreviations – used to circumvent censorship, demonstrating a higher detection rate than traditional, less-sophisticated methods.

Testing with the GPT-4 model showed a 66.46% accuracy in identifying illegal content and 77.09% accuracy in detecting harmful content, indicating its potential for real-world application. Dr. Baek Jong-woo stated that the AI-based system “will be useful in creating a safe digital environment and early detection of high-risk groups,” and expressed hope that it will serve as a foundation for building a national suicide prevention infrastructure.

The development comes as South Korea grapples with a continuing crisis in suicide rates, despite efforts to address the issue. Recent data indicates a need for improved support for individuals after they leave hospital care, with a 25-fold increase in suicide risk observed among those discharged from treatment.