In the first blog in our artificial intelligence (AI) series, we explored why people are turning to AI for mental health support. In our survey, we found that most people who responded had used an AI platform for help with their mental health or wellbeing – noting that AI can be an accessible, low cost, non-judgemental option. These factors can make artificial intelligence (AI) appealing, especially when human support can be overstretched or difficult to access. 

But alongside its potential benefits, people with lived experience raised important questions about the limitations and risks of leaning on AI for mental health support. 

One person told us: 

“Sometimes it can be TOO validating and collude with my perspective when my perspective is not good or helpful to me. This could potentially be dangerous.” 

This begs the question: What would happen if someone was having suicidal thoughts or thoughts of self-harm and was encouraged to act on these? What if someone was experiencing psychosis, and their beliefs or hallucinations were encouraged? The encouragement could have devastating implications for people seeking help.   

People with lived experience told us their biggest concerns when it comes to the use of AI for mental health support. We’ve broken it down into three main worries. 

1. AI cannot understand the whole person 

One person said: “It cannot read my non-verbal cues, my tone of voice, my facial expression, and it cannot ‘hold space’ for me when I’m really suffering. It cannot co-regulate with my nervous system as effectively as an attuned human being.” 

The disconnect between humans and artificial intelligence can leave people receiving blunt and unempathetic responses, which often wouldn’t be the case with professional, therapeutic support. 

2. Helpful information can still be wrong 

Another person with lived experience said: “I have read stories about people getting obsessed with it, overusing it, relying totally on it, rather than questioning anything that doesn’t seem right. In some situations, it can easily be led into giving bad advice, especially when it tries to just tell you what it ‘thinks’ you want to hear.” 

AI’s confident responses are not a replacement for informed guidance and support from a trained mental health professional. Often this technology can be agreeable but does not always supply correct advice or signpost to appropriate support. 

3. AI must not become a substitute 

In our previous blog, we discovered that people may use AI because of long service waiting lists, costs, or gaps in mental health services, which leave them with fewer alternative options.  

The danger of this is that an emergency response to unmet need becomes accepted as a replacement for properly funded human support. We know that mental health charities provide services to over eight million people in England alone and their expert advice, guidance and support cannot be easily replicated. This concern was raised by many of our respondents with lived experience, who said: 

“We will lose the human touch. An AI will never understand the complexity of human nature and the way mental illness gets at the core of it.” 

“It can miss nuance around lived experience, neurodivergence or trauma. There’s also a limit to how much support it can give compared to a real person, particularly in more serious or emotional situations.” 

These concerns are also emerging across the wider mental health sector.  

This is another important opportunity to ensure that decisions about AI are shaped by the people most affected by it.  
 
As AI continues to rapidly develop in its ability to replicate human relationships, think (or process) independently and, in some cases, take on a life of its own, global regulations on the development and use of AI are even more necessary. 

People with lived experience of poor mental health and illness should inform: 

• What safe and unsafe use looks like 

• What information people need about an AI tool’s limitations 

• When someone should be directed towards human or crisis support 

• How consent, privacy and transparency should work 

The greatest risk may not be that people consciously choose AI instead of human services. It may be that overstretched services leave them with little choice and turn them towards a largely unregulated platform. 

AI may have a valuable role in supporting mental health, but innovation cannot come at the price of safety, accountability or human connection. The findings from our survey show why people with lived experience must be involved from the beginning.