IBM’s Watson could need a little more time in medical school, if a new report is accurate.
Documents reviewed by Stat, a medical news site, show the artificial intelligence supercomputer has shown “multiple examples of unsafe and incorrect treatment recommendations” for cancer patients.
The allegations, it’s worth noting, were made two years ago and no patients were harmed.
Incomplete training is being cited for the gaps, says the report. IBM researchers and doctors at the Memorial Sloan Kettering Cancer Center educated the system using hypothetical cases, rather than actual patient data.
IBM, though, takes issue with the claims, saying “it’s not an accurate or timely representation.”
“Today, Watson for Oncology is trained to help oncologists treat 13 cancers,” the company said. “Our oncology and genomics offerings are used by 230 hospitals around the world and have supported care for more than 84,000 patients, which is almost double the number of patients as of the end of 2017. … We have learned and improved Watson Health based on continuous feedback from clients, new scientific evidence and new cancers and treatment alternatives. This includes 11 software releases for even better functionality during the past year, including national guidelines for cancers ranging from colon to liver cancer.
“We remain absolutely committed to Watson Health to give providers and professionals the technology and expertise to help transform health for people everywhere.”
The report comes a little over a year after the American Society of Clinical Oncology gave Watson high marks for its cancer treatment plan suggestions.
AI training is one of the most challenging aspects for Watson. To adequately adapt for various scenarios, the system must learn nuance, which requires a lot of information and often in unusual ways. For example, one test the system undergoes, though not necessarily for medical testing, is called the “muffin challenge,” where Watson must differentiate between the tops of a blueberry muffin or the picture of a curled up Chihuahua.
The system has beaten most humans in that test.