The use of Watson for oncology is attracting the glare, not warmth, of the spotlight. Numerous tech watching sites have covered a July 25 STAT report over internal documents which indicated criticism of the Watson for Oncology system.
Yes, that STAT report said the Watson supercomputer had at times sent out erroneous cancer treatment advice. Company medical specialists and customers identified examples of incorrect treatment recommendations as IBM was promoting the product to hospitals and physicians. Kaiser Health News' Morning Briefing on the 25th also told its readers that documents showed computer recommendations that were flawed.
What could possibly go wrong with a supercomputer handling oncology topics?
The reports point to the way in which Watson was trained for this. "The software was drilled with a small number of 'synthetic' cancer cases, or hypothetical patients, rather than real patient data," said KHN.
Mariella Moon in Engadget also noted the training factor. She said engineers and doctors fed Watson hypothetical patients' data and treatment recommendations by MSK doctors instead of real patients' information.
Chris Merriman, the INQUIRER, weighed in, delivering a bigger-picture of man versus machine: "nobody is saying that this is proof that AI doesn't work in medicine. But it's early days and this incident reflects one of the biggest concerns in the field, and it's one that has affected every part of computing since Babbage and Lovelace. GIGO - Garbage in, Garbage out."
In fairness, however, it is necessary to look further into Watson for oncology purposes. There is another side to this story and it was addressed by Jennings Brown in Gizmodo.
A Memorial Sloan Kettering (MSK) Cancer Center spokesperson spoke to Stat about a drug called into question because of Watson recommending it. Yet, because of the patient's own condition, it was a dangerous drug for the patient to be taking. "A Memorial Sloan Kettering (MSK) Cancer Center spokesperson told Stat that they believed this recommendation was not given to a real patient, and was just a part of system testing."
Meantime, here is how the MSK has described their involvement: "Watson Oncology is a cognitive computing system designed to support the broader oncology community of physicians as they consider treatment options with their patients. Memorial Sloan Kettering clinicians and analysts are partnering with IBM to train Watson Oncology to interpret cancer patients' clinical information and identify individualized, evidence-based treatment options that leverage our specialists' decades of experience and research."
Brown, senior editor and reporter at Gizmodo , said "a IBM spokesperson told Gizmodo that Watson for Oncology is trained to help oncologists treat 13 cancers and is being used by 230 hospitals around the world, and has "supported care for more than 84,000 patients."
Oncologists at the VA are reported to be using IBM Watson for Genomics technology to help them interpret cancer data in the treatment of veteran patients through the VA's precision oncology program. The latter primarily supports stage 4 cancer patients who have exhausted other treatment options.
Another reason to put the criticism in perspective is that IBM treats Watson for Oncology as a continued effort, not a done and dusted solution. An IBM spokesperson was quoted in Gizmodo's report as saying Watson has had multiple software releases including national guidelines for cancers:
"At the same time, we have learned and improved Watson Health based on continuous feedback from clients, new scientific evidence, and new cancers and treatment alternatives," the spokesperson said. "This includes 11 software releases for even better functionality during the past year, including national guidelines for cancers ranging from colon to liver cancer."