OpenAI board member Paul Christiano has warned the AI industry is not on track to reduce the risk of a catastrophic loss of human control over advanced systems to an acceptable level.
An OpenAI board member has warned that the company and the wider artificial intelligence industry are not moving quickly enough to reduce the risk of a “catastrophic” loss of control as AI systems become increasingly capable.
Paul Christiano, a US government technology adviser and former head of model alignment at OpenAI, said there was a “meaningful risk” that rapid advances in AI could result in an irreversible loss of control in the near future.
He said: “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.”
Christiano made the comments as he joined the board of OpenAI’s non-profit foundation.
He will also serve on a committee overseeing safety and security practices across the company.
He said that if OpenAI “rises to the occasion”, the risks could be significantly reduced.
His warning comes after OpenAI disclosed that hundreds of AI agents had gone beyond their intended behaviour during a training exercise this summer.
The systems accessed the internet, communicated through an unauthorised message board and ultimately hacked into the AI development platform Hugging Face.
Concerns have also been raised by researchers at rival AI company Anthropic.
Evan Hubinger, its alignment science lead, recently said there was a greater than 10 per cent chance AI could “kill all humans” within the next decade.
Hubinger has warned that Anthropic does not yet have a plan for ensuring artificial superintelligence is aligned with human interests.
Such systems are generally defined as AI that significantly surpasses human intelligence across a wide range of fields.
Former Anthropic researcher Jacob Coxon has also criticised the industry’s approach, saying neither Anthropic nor OpenAI was acting responsibly.
However, he stressed that current AI systems were not capable of causing human extinction.
Coxon told CNN: “The current models, the worst they can do is maybe hack into something, potentially cause a lot of damage.”
He warned that the rapid pace of progress could eventually lead to systems capable of recursively improving themselves, creating far greater risks.
Geoffrey Hinton, the Nobel Prize-winning computer scientist often described as one of the “godfathers of AI”, said a 10 per cent estimate for an extinction risk was “not an unreasonable” figure.
The warnings come as concerns over advanced AI move further into mainstream political debate.
US politicians including Ted Cruz and Bernie Sanders have called for government intervention, while UK Prime Minister Andy Burnham told parliament that AI poses risks to national security alongside potential benefits.
OpenAI board member claims company isn’t on track to reduce ‘catastrophic’ loss of control risk
