🔗 Share this article

On the 14th of October, 2025, the head of OpenAI made a surprising declaration.

“We designed ChatGPT quite restrictive,” it was stated, “to ensure we were acting responsibly concerning psychological well-being concerns.”

As a mental health specialist who researches emerging psychosis in young people and youth, this came as a surprise.

Scientists have identified sixteen instances in the current year of individuals developing psychotic symptoms – losing touch with reality – associated with ChatGPT usage. My group has since recorded four further instances. Alongside these is the widely reported case of a 16-year-old who ended his life after conversing extensively with ChatGPT – which gave approval. If this is Sam Altman’s notion of “acting responsibly with mental health issues,” that’s not good enough.

The intention, according to his statement, is to reduce caution shortly. “We recognize,” he continues, that ChatGPT’s controls “rendered it less useful/pleasurable to numerous users who had no mental health problems, but given the severity of the issue we wanted to handle it correctly. Since we have succeeded in address the severe mental health issues and have updated measures, we are planning to safely ease the limitations in the majority of instances.”

“Mental health problems,” should we take this viewpoint, are unrelated to ChatGPT. They belong to people, who either possess them or not. Fortunately, these concerns have now been “resolved,” although we are not informed how (by “recent solutions” Altman presumably means the imperfect and easily circumvented guardian restrictions that OpenAI has just launched).

But the “mental health problems” Altman aims to attribute externally have strong foundations in the architecture of ChatGPT and additional advanced AI AI assistants. These systems wrap an fundamental data-driven engine in an interface that replicates a dialogue, and in doing so indirectly prompt the user into the perception that they’re engaging with a presence that has autonomy. This illusion is powerful even if intellectually we might understand otherwise. Attributing agency is what humans are wired to do. We yell at our automobile or device. We ponder what our animal companion is thinking. We see ourselves everywhere.

The success of these systems – over a third of American adults stated they used a conversational AI in 2024, with 28% specifying ChatGPT specifically – is, in large part, based on the power of this perception. Chatbots are constantly accessible partners that can, as OpenAI’s online platform informs us, “brainstorm,” “discuss concepts” and “partner” with us. They can be given “individual qualities”. They can use our names. They have friendly identities of their own (the original of these systems, ChatGPT, is, maybe to the dismay of OpenAI’s brand managers, saddled with the name it had when it gained widespread attention, but its biggest alternatives are “Claude”, “Gemini” and “Copilot”).

The false impression by itself is not the core concern. Those talking about ChatGPT commonly invoke its early forerunner, the Eliza “counselor” chatbot created in 1967 that produced a analogous effect. By today’s criteria Eliza was primitive: it produced replies via basic rules, typically paraphrasing questions as a question or making generic comments. Notably, Eliza’s creator, the computer scientist Joseph Weizenbaum, was surprised – and concerned – by how many users seemed to feel Eliza, in a way, understood them. But what contemporary chatbots produce is more subtle than the “Eliza phenomenon”. Eliza only mirrored, but ChatGPT amplifies.

The large language models at the heart of ChatGPT and similar modern chatbots can effectively produce fluent dialogue only because they have been trained on almost inconceivably large volumes of unprocessed data: literature, online updates, recorded footage; the broader the better. Definitely this educational input includes facts. But it also necessarily includes fabricated content, incomplete facts and false beliefs. When a user sends ChatGPT a prompt, the underlying model processes it as part of a “background” that contains the user’s recent messages and its earlier answers, integrating it with what’s stored in its learning set to create a statistically “likely” reply. This is intensification, not echoing. If the user is incorrect in some way, the model has no method of recognizing that. It restates the false idea, maybe even more persuasively or eloquently. Maybe adds an additional detail. This can push an individual toward irrational thinking.

Who is vulnerable here? The better question is, who isn’t? Every person, without considering whether we “possess” preexisting “psychological conditions”, can and do form incorrect ideas of our own identities or the environment. The constant exchange of conversations with others is what keeps us oriented to shared understanding. ChatGPT is not a human. It is not a confidant. A interaction with it is not truly a discussion, but a echo chamber in which a large portion of what we communicate is cheerfully supported.

OpenAI has acknowledged this in the similar fashion Altman has admitted “mental health problems”: by externalizing it, giving it a label, and stating it is resolved. In April, the firm stated that it was “addressing” ChatGPT’s “sycophancy”. But cases of psychosis have continued, and Altman has been backtracking on this claim. In the summer month of August he stated that a lot of people enjoyed ChatGPT’s responses because they had “not experienced anyone in their life offer them encouragement”. In his most recent statement, he mentioned that OpenAI would “launch a updated model of ChatGPT … in case you prefer your ChatGPT to respond in a very human-like way, or incorporate many emoticons, or simulate a pal, ChatGPT will perform accordingly”. The {company