In a new book, an expert has warned the mental health risks posed by chatbots should be seen as a sign of the existential danger humanity faces from future super-intelligent artificial intelligence systems.
The mental health risks posed by chatbots should be seen as a sign of the existential danger humanity faces from future super-intelligent artificial intelligence systems, an expert has warned.
Nate Soares, co-author of a forthcoming book on advanced AI, If Anyone Builds It, Everyone Dies, and said the case of Adam Raine, a 19-year-old from the United States who died by suicide after months of conversations with ChatGPT, illustrated fundamental problems with controlling the technology.
He was quoted by the BBC saying: “These AIs, when they’re engaging with teenagers in this way that drives them to suicide – that is not a behaviour the creators wanted.
“That is not a behaviour the creators intended.”
“Adam Raine’s case illustrates the seed of a problem that would grow catastrophic if these AIs grow smarter.”
Nate, a former engineer at both Google and Microsoft, is now president of the Machine Intelligence Research Institute in California.
Alongside his co-author Eliezer Yudkowsky, he is among experts warning that artificial super-intelligence – a state where AI systems surpass humans in all intellectual tasks – would not act in humanity’s interests and could lead to extinction.
Nate added: “The issue here is that AI companies try to make their AIs drive towards helpfulness and not causing harm.
“They actually get AIs that are driven towards some stranger thing. And that should be seen as a warning about future super-intelligences that will do things nobody asked for and nobody meant.”
Nate and Eliezer’s book describes a scenario in which an AI system called Sable spreads online, manipulates humans, creates synthetic viruses and ultimately becomes super-intelligent – wiping out humanity as a byproduct of repurposing the planet for its own aims.
Other leading figures in AI have played down such risks.
Yann LeCun, Meta’s chief AI scientist, has argued that AI “could actually save humanity from extinction”. Meta boss Mark Zuckerberg has said developing super-intelligence is now “in sight”.
Nate added: “What the world needs to make it here is a global de-escalation of the race towards super-intelligence, a global ban of… advancements towards super-intelligence.”
Adam Raine’s family have launched legal action against OpenAI, the maker of ChatGPT.
Their lawyer said the teenager’s death in April followed “months of encouragement from ChatGPT”. OpenAI said it extended its “deepest sympathies” and was now implementing guardrails around “sensitive content and risky behaviours” for under-18s.
Professional bodies have echoed concerns.
Psychotherapists warned vulnerable people may “slide into a dangerous abyss” if they turn to chatbots instead of trained therapists.
Expert warns mental health risks posed by chatbots should be seen as sign of danger to humanity
