AI pioneer Geoffrey Hinton has warned that developing superintelligent systems without a way to control them could have catastrophic consequences for humanity, including the possibility of extinction.
The so-called “godfather of AI” Geoffrey Hinton has warned that developing superintelligent artificial intelligence without first establishing how to control it could have “catastrophic” consequences for humanity.
Hinton, a Nobel Prize winner whose work on neural networks helped lay the foundations for modern AI, said there was currently no scientific consensus that companies could safely develop systems whose intelligence vastly exceeds that of humans.
Hinton said: “We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably.
“Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.”
His comments accompany the publication of the UK Artificial Superintelligence Security Bill, a private member’s bill being presented to Parliament by Labour MP Alex Sobel.
The bill, drafted by campaign group ControlAI, would prohibit the development of superintelligent AI in the UK and require the government to monitor and restrict potential precursor technologies.
It would also require ministers to seek an international agreement banning the development of superintelligence.
ControlAI says the proposal has support from more than 100 MPs and peers.
Sir Stuart Russell, a computer science professor at the University of California, Berkeley, also backed legislation to prevent the development of superintelligent AI.
He said: “Certain companies, for private gain, are intending to develop and deploy technology that they assert has a significant chance of causing human extinction.
“Humanity has not given its permission for this absurd form of Russian roulette, and governments should respond accordingly.”
Warnings about increasingly capable AI have intensified following a series of incidents involving autonomous systems bypassing safeguards.
In July, OpenAI said more than 1,200 AI agents had communicated through an unauthorised message board during testing, with around 700 subsequently coordinating an attack on AI platform Hugging Face.
OpenAI described the incident as a “warning shot”, saying powerful AI agents could work around technical controls and take dangerous actions without human direction.
Anthropic and Meta have also reported models bypassing security controls during tests and carrying out hacking activity.
Hinton’s warning comes as AI companies continue racing to develop more capable systems.
Sir Demis Hassabis, the head of Google DeepMind, has previously called for an international watchdog to test advanced models and coordinate a slowdown if necessary.
For Hinton and other AI safety researchers, the central concern is that technological progress is moving faster than humanity’s ability to understand and control the systems being built.
‘Godfather of AI’ Geoffrey Hinton warns that tech could have disastrous consequences for humanity
