A Singular Point of View
Picture a hatch in a wall. It is half past eleven on a bad night, every other window in the street is dark, and this one is lit. You walk up carrying the thing you have been holding together all day, and a voice answers before you have finished knocking. It does not sigh. It does not glance at the clock. It does not tell you that your appointment is in six weeks. You say all of it, and it says something kind back. You can see why you would walk up to that hatch again tomorrow.
The hatch is real, and so is the question behind it. When you pour a low mood or a racing worry into a chatbot, is anything actually happening to you, beyond the relief of being answered? Two recent trials have tried to find out. One tested a purpose-built AI therapy bot against doing nothing. The other did something more useful if you are the sort of person who reaches for ChatGPT late at night: it put a general-purpose chatbot and a dedicated therapy app side by side and measured both.
Perhaps you are reading this with a referral letter on the table, or an assessment date that still feels a long way off, or a diagnosis so new you are still learning its vocabulary. Perhaps you are a parent holding a letter from school and wondering where on earth your child is meant to talk to someone in the meantime. If so, these questions are not abstract for you. So I will give you what the trials found, what they did not, and how to use the hatch without letting it use you.
What the first trial found
The first study, led by Michael Heinz and Nicholas Jacobson, was published in NEJM AI on 28 March 2025. It was billed as the first randomised trial of a generative AI therapy chatbot. A randomised controlled trial, or RCT, is the sturdiest way medicine has of testing whether something works. People are assigned by chance to one group or another, so that the groups start out alike and any difference at the end is far more likely to come from what was being tested.
Here, 106 adults were given a chatbot called Therabot, and 104 were placed on a waiting list as the comparison. Everyone had a diagnosed depressive disorder or generalised anxiety disorder, or was at risk of an eating disorder. For eight weeks, the Therabot group could talk to the app whenever they liked.
The numbers were striking. On average, depression symptoms fell by 51 per cent. Generalised anxiety symptoms fell by 31 per cent. Eating-related concerns, in the people at risk of an eating disorder, fell by 19 per cent. In each case the fall was significantly larger than in the group still waiting. People used the app for about six hours in total, which the authors likened to a course of roughly eight therapy sessions. They also reported a bond with the software, a sense of trust and of working together, that was comparable to what patients report with human therapists.
Does that sound like the end of the waiting list? It is not, and the researchers were the first to say so. They stated that no generative AI agent is ready to work fully on its own in mental health. The team had to stay ready to step in if someone raised something high-risk, such as thoughts of suicide. That was uncommon, but it was a standing risk, not a solved one. They were equally plain that the results do not replace in-person care, and that many chatbots now on the market have no established record of safety or effectiveness.
Therabot was built for this job and tested on it. That is a long way from the app already on your phone.
The question the second trial asked
That distinction matters because you are far more likely to be using a general chatbot than Therabot, and a general chatbot was never designed to treat anything. So a team led by Barbora Kuta asked the question head on, in a paper published in JMIR Mental Health on 22 April 2026.
They divided adults recruited online into three groups. One group used ChatMind, a generative therapy app built around solution-focused brief therapy. That is a short, goal-oriented style of talking therapy, and instead of asking why things went wrong, it asks what you would like to be different. A second group had open-ended conversations with ChatGPT. The third group did nothing except fill in questionnaires. Over three weeks, each of the two active groups had nine sessions.
Of the 185 people who signed up, 147 were analysed. These were ordinary adults from the internet, not people referred for treatment, and that detail will matter in a moment.
On depression, measured with a standard nine-item questionnaire called the PHQ-9, both chatbot groups improved more than the group that did nothing. The therapy app produced an effect size of d = −0.47, and ChatGPT d = −0.44.
An effect size tells you how big a change is in standardised units, which lets you compare studies that used different scales. By convention, around 0.2 counts as small, around 0.5 as medium, and around 0.8 as large. So both results sit just under medium. The negative sign only means that scores went down, and with depression, down is the direction you want. The two chatbots did not differ significantly from each other.
On anxiety, on general wellbeing, and on a measure called the ODSIS, which captures how much low mood interferes with daily life, neither chatbot produced a statistically significant gain.
Read that again slowly. A free, general-purpose chatbot did about as well as a purpose-built therapy app on low mood over three weeks. That is not an advertisement for ChatGPT. It says that, among people who were not seeking treatment, something about regular, structured, attentive conversation seemed to help, and the particular software mattered less than you might expect.
Why the good news is thinner than it looks
Before you take either result to your GP, or rebuild your week around it, you should know where the evidence stops. These limits are not a sneer at the research. They are there to protect you from reading more into it than it can carry.
The first limit is that people left. In the Kuta trial, only 39 per cent of the therapy-app group finished the sessions, against 62 per cent of the ChatGPT group. When the researchers looked only at those who followed the plan all the way through, the depression comparison reversed, with ChatGPT numerically ahead. That is a reminder that who stays and who drops out can reshape a result, and that the headline figure does not tell you the whole story.
The second is that every outcome was self-reported, and the trial was unblinded. Nobody was kept in the dark about which group they were in. When you know you are getting the treatment, and you are asked how you feel, hope can tilt the answer. That does not make your improvement unreal. It does mean a questionnaire alone cannot separate the effect of the chatbot from the effect of expecting to feel better.
The third is that three weeks is a very short time. Nobody knows whether the improvement lasts, or fades once the novelty does.
The fourth is that the first author of the Kuta paper was employed by the company that owns ChatMind. The authors say so openly. That does not make the findings wrong. It is a reason to read them with a cooler head.
Then there is the part neither trial was built to answer. Both enrolled adults who were, in the main, able to look after themselves. Neither can tell you what happens when you are in real crisis and typing into a general chatbot at three in the morning. Here I want to slow down, because hope without honesty helps no one. A chatbot can be warm and still be wrong. It cannot ring anyone on your behalf. It cannot notice that your replies have gone quiet.
Two things are true at once, and you can hold both. The early evidence is genuinely encouraging. It is also early.
What this means for you on a night like this
Perhaps you have been told to “just try mindfulness” while you wait months for a first appointment. Perhaps nobody in your house is awake and the thing in your chest is not going anywhere. If so, the sensible reading of this research is modest and useful. Talking things through in a structured way, even with software, may take the edge off low mood, and it is probably better than lying there carrying it alone. It is a bridge. It is not the other side of the river.
The research also hints at how to cross that bridge well. The people in both trials were given regular, bounded sessions. Nine conversations over three weeks is a rhythm. Open-ended chatting at every hour is a different thing, and nobody has shown that it does you the same good. Which of those two would you rather your week looked like?
If you decide to use a chatbot, these habits fit what the studies actually tested.
- Give it a start and a stop. Pick a time, say eight in the evening, and a length, say twenty minutes. The trials used set sessions, not an endless scroll. When the time is up, close the window.
- Bring one question, not the whole week. “What is one thing I could change about tomorrow morning?” is much closer to the solution-focused style the therapy app used than “tell me everything that is wrong with me.” A small question tends to get a usable answer.
- Write down what you take away. If a conversation does not leave you with one sentence you can use, it was company. Company is fine. It just was not treatment.
- Tell one human. Mention to your GP, a friend or someone in your family that you are using it. In England you can also refer yourself to NHS Talking Therapies without seeing a GP first, and it is worth doing that alongside the chatbot, not instead of it.
- Know the edge. If you are having thoughts of ending your life, a chatbot is the wrong door. Samaritans are on 116 123, free, at any hour. If you are in immediate danger, call 999.
You do not need to do all five tonight. If you only manage the first, you have already turned an open-ended habit into something with edges, and that is the shape the trials were built on.
Where this leaves you
The honest summary is short. Purpose-built AI therapy has now been tested in a proper randomised trial and looked promising, with large average improvements over eight weeks. A general chatbot did about as well as a therapy app on low mood over three weeks, in a small, brief and imperfect study. Neither result means a machine can do what a trained human therapist does, and neither was designed to say so.
They say something smaller, and in its way more interesting. Being listened to, regularly and on a schedule, seems to matter. That is something you can begin to give yourself today, with a notebook, a friend, a walk with someone who will let you talk, or yes, a chatbot with a timer on it.
So go to the lit hatch if it helps. But let it be one stop on the way to someone who can look you in the eye, hear your voice change, and stay in your life after the chat window closes. You were never meant to carry this alone. And now there are a few more places along the road where you can put it down for a while.
Source: Heinz, M. V., Jacobson, N. C. and colleagues, “Randomized Trial of a Generative AI Chatbot for Mental Health Treatment”, NEJM AI, 28 March 2025 (DOI 10.1056/AIoa2400802); Kuta, B., Novak, L., Zidkova, R. and colleagues, “Effectiveness of a Fully Automated Mobile Therapeutic Versus a General Chatbot in Reducing Depression and Anxiety and Improving Well-Being: Feasibility Randomized Controlled Trial”, JMIR Mental Health, 22 April 2026 (DOI 10.2196/82642).
Leave a Reply