Preventing AI Hallucinations in Psychiatric Clinical Decision Support Systems
Artificial intelligence is emerging as a tool in psychiatric care, with systems designed to support clinical decisions while guarding against hallucination.
- Christoph Correll detailed a clinical decision support system linked to electronic health records that pulls patient histories and ambient speech to update diagnoses and treatment plans in real time.
- A 2025 review published in Artificial Intelligence in Medicine identified 84 regulated AI-enabled tools for mental health care, finding that only 7 held regulatory clearance and merely 5 possessed peer-reviewed validation studies.
- To prevent hallucination risks, the Comentra platform under development by MedLink Global is barred from offering recommendations when data is insufficient, instead prompting clinicians with specific diagnostic questions.
Guarding Against Algorithmic Hallucinations in Clinical Decision Support
As psychiatric practices evaluate emerging digital infrastructure, Christoph Correll described a clinical decision support system designed to integrate with electronic health records. The platform pulls available patient histories, intake data, and ambient speech from clinical visits to update suggested diagnoses and treatment plans dynamically. During or following a session, clinicians review two or three options, such as pharmaceutical augmentation for depression. Correll serves as chief medical officer of MedLink Global, a Mayo Clinic Platform_Accelerate–incubated company developing this AI-based clinical decision support platform, known as Comentra.
The primary developmental goal for the platform is demonstrating superior performance compared to usual care, which necessitates showing it does not hallucinate. When input information is insufficient, the system is strictly barred from generating recommendations. Instead, it formulates targeted clinical questions to bridge informational gaps—such as inquiring about hepatic problems, mania, or histories of suicidality—prior to offering guideline-based options. Correll emphasized that these mechanisms assist clinicians in securing a comprehensive view of historical and present clinical needs to drive measurement-based care.
Validating Patient Outcomes Across Large-Scale Cohorts
Correll cited collaborative work with the Mayo Clinic encompassing approximately 40,000 patients, noting that individuals whose clinicians had made the choices the decision system now suggests experienced better clinical outcomes. However, broader clinical uptake remains an open question requiring educational initiatives so practitioners can effectively evaluate, select, or deselect system recommendations. Researchers are also investigating whether off-label treatments backed by supporting clinical data yield better outcomes for specific patient subsets, even though standard evidence-based decision support systems would not typically flag them.
Correll noted that artificial intelligence should not make ultimate treatment decisions, but rather provide a complete data overview to inform expert judgment. This cautious integration reflects a wider validation deficit across the specialty. A 2025 review of regulated artificial intelligence-enabled clinical decision support tools for mental health care, published in Artificial Intelligence in Medicine, identified 84 products. Of those, only seven held regulatory clearance from the United States Food and Drug Administration or European and UK authorities, and just five peer-reviewed validation studies supported those seven tools. The review authors cautioned that regulatory approval does not guarantee external validity, emphasizing the need for standardized reporting.
Advancing Toward Phenotype-Driven Precision Psychiatry
Exploring the potential for population-level comparisons, Correll stated that artificial intelligence could advance the field toward precision psychiatry, provided that patient phenotypes are systematically measured. Natural language processing of clinician notes serves as one data source, though its utility depends on whether symptoms were documented. Alternatively, standardized five- to ten-minute avatar interviews conducted prior to visits could supply more consistent symptom data. With standardized data inputs, models could estimate whether initiating specific selective serotonin reuptake inhibitors, serotonin-norepinephrine reuptake inhibitors, or atypical antipsychotic augmentations correlates with reduced hospitalizations or lower suicidality for individual patients. Correll characterized such outcome modeling as operating more like a black box than standard guideline recommendations, stating that its reliability depends directly on systematic measurement feeding the underlying algorithms.
Disclaimer: The information provided in this article is for educational and scientific communication purposes only and does not constitute medical advice. Always consult with a qualified healthcare provider regarding any medical condition, diagnosis, or treatment plan.