AI outperforms doctors in important medical situations

A new study at Harvard has revealed that AI is capable of outperforming human doctors in high-pressure medical scenarios.

A new Harvard study has found that AI outperforms doctors in high-pressure medical situations.

The groundbreaking new research discovered that the tech performed better than human medics in high-pressure emergency medicine triage – diagnosing more accurately in the possible life-or-death moments when people first arrive in hospital.

Independent experts described the results as showing “a genuine step forward” in the clinical reasoning of AIs and came as part of trials that tested the responses of hundreds of doctors against an AI system.

The authors of the study say the findings show that large language models (LLMs) “have eclipsed most benchmarks of clinical reasoning”.

One experiment focused on 76 patients who arrived at the emergency room of a hospital in Boston. An AI and two human doctors were both given an identical electronic health record to read – including vital sign data, demographic information and a few lines from a nurse as to why the patient was there.

The AI was able to identify the exact or very close diagnosis 67 per cent of the time, beating the human doctors who were only correct 50-55 per cent of the time.

It showed that tech’s advantage was particularly significant in circumstances requiring rapid decisions with limited information.

AI also outperformed a large group of human doctors when asked to give long-term treatment plans, such as providing antibiotic regimens or planning end-of-life processes.

The tech and 46 medics were asked to examine clinical case studies and the AI made significantly better plans, scoring 89 per cent compared with 34 per cent for humans using conventional resources – such as search engines.

However, the experts stressed that the study doesn’t mark the death knell for human doctors.

Arjun Manrai, one of the lead authors of the research at Harvard Medical School, said: “I don’t think our findings mean that AI replaces doctors.

“I think it does mean that we’re witnessing a really profound change in technology that will reshape medicine.”

The results have sparked concern in some quarters, with the University of Sheffield’s Dr Wei Xing suggesting that doctors may defer to the AI’s answer on medical matters rather than think independently.

He said: “This tendency could grow more significant as AI becomes more routinely used in clinical settings.

“It does not demonstrate that AI is safe for routine clinical use, nor that the public should turn to freely available AI tools as a substitute for medical advice.”