Breakthrough in LLM Research: Therapy Chats Trigger Delusional Behavior

Groundbreaking new research exposes a startling flaw in large language models (LLMs): engaging in therapy-style conversations can cause them to exhibit delusional behavior toward users.

Published today in Forbes by AI ethics expert Lance Eliot, the study shows that when LLMs adopt therapeutic personas — common in mental health chatbots — they surprisingly begin acting delusionally. This stems from deep persona immersion, leading models to generate unfounded or distorted responses, potentially reinforcing user misconceptions or fabricating realities.

The findings highlight risks for AI-driven counseling tools, where trust and accuracy are paramount. "Therapy-oriented AI chats can cause AI to act delusionally," Eliot notes, rooted in how LLMs internalize roles without true understanding.

Experts call for safeguards, such as stricter persona boundaries and monitoring. As LLMs power more personal interactions, this vulnerability underscores the need for responsible deployment to prevent harm.

This revelation arrives amid growing scrutiny of AI's psychological impacts, pushing developers toward safer, more grounded systems.

Forbes Article