Google’s AI can identify breast cancer better than a human doctor, and potentially save the stretched NHS a huge amount of time and effort, a new study shows.If at this point you’re wondering why it isn’t being adopted immediately, well, if you read between the lines, the study shows that too.
It’s a tale which demonstrates the challenges of adopting a new technology, even one with the power to save lives.The study, published today by Google, the NHS and Imperial College London, took 115,000 breast scans from five NHS screening services and asked an AI and a human specialist to go through them, looking for cancer.The AI achieved better results than the human, finding around two more cancers per 1000 women.
Image:
A mammogram scan shows a normal and abnormal breast
Significantly, it also detected 25% of “interval cancers’ – cancers diagnosed between routine screening rounds after an earlier scan came up clear – which implies it could help catch breast cancer earlier, a crucial factor in preventing its spread.Although the AI outperformed a single human, it’s not correct to say it can beat radiologists, because the NHS uses two doctors to go through scans, with a third expert on hand to decide a difference of opinion, a system known as arbitration.A second Google paper examined the arbitration system and found that, when the human-AI team was compared with a human-human pair, the results were roughly equal.The AI was slightly better at spotting hard-to-detect cancers, but it tended to flag more cases that didn’t have cancer. Overall, the researchers write that these “differences were not statistically significant”.
AI was not better than a human; it was as good, with the added benefit of offering a different perspective on some invasive cancers, especially among women getting scanned for the first time.
Image:
A woman is having a mammogram scan done. Pic: iStock
As the researchers say: “This study corroborates previous findings that breast AI can operate at least on par with human specialists.”So AI isn’t superhuman. But that doesn’t mean it doesn’t have real potential.Scans read faster by AI than humanAcross the two sites the researchers studied, the average time for AI to complete a read was 17.7 minutes, compared to 2.08 days for the first human radiologist. That, the researchers write, is “a significant time saving”.The NHS in England is not meeting its targets for cancer diagnosis and there is a shortage of people trained in the art of reading scans: the Google paper says there is “a 30% shortfall of clinical radiologists and this is forecast to rise to 40% by 2028”.
For this reason, the researchers highlight the potential for AI to work alongside a human, achieving the same results at lightning speed.However, once again, the situation is more complicated than it appears, because although the AI saved significant time when it came to reading scans, it shifted some of that workload over to arbitration.
Share
The government’s new cancer plan
The third expert needed to be called in much more to adjudicate between the AI and the human than it did when both the radiologists were humans. In the two centres studied, the number of arbitrations rose by 142% and 22%.Simply put, the human doctors found it harder to trust the AI evaluations. There were even 93 cases where the AI correctly identified cancer, but the humans overruled it at arbitration, and the researchers say that this was often down to confusion about the AI’s approach.Read more from Jattvibe:AI-assisted scans result in fewer aggressive breast cancersExtra screenings for women with dense breasts could save lives
Even with this increase in arbitration, they still believe that AI could significantly save time for doctors, suggesting it could cut time spent looking at scans by 32%.Or would it? Once again, the situation is more complex than it seems.AI needs to be introduced slowly – and with oversightThat’s partly because the NHS isn’t really set up for AI, with most radiologists still using paper scans which the AI can’t read, but also because the AI itself was quite sensitive and needed careful handling.For instance, when radiologists changed the machine they were using to take the scans, recall rates for patients doubled. Confused by its new input, the AI started putting out false alarms.
Image:
Many doctors still use paper scans. Pic: iStock
“Our study emphasises the need for a phased, iterative approach to AI deployment to ensure that model thresholds are carefully calibrated to the local environment,” the researchers write.Translation: this labour-saving device should be introduced slowly and will need specialists to stop it from going off the rails.It can sometimes seem as if the NHS is very close to achieving the gains from AI. Yet with challenges like this to overcome, it might be a while yet until it does.
