SoylentNews is people
SoylentNews
AI techniques in medical imaging may lead to incorrect diagnoses:
A team of researchers, led by the University of Cambridge and Simon Fraser University, designed a series of tests for medical image reconstruction algorithms based on AI and deep learning, and found that these techniques result in myriad artefacts, or unwanted alterations in the data, among other major errors in the final images. The effects were typically not present in non-AI based imaging techniques.
The phenomenon was widespread across different types of artificial neural networks, suggesting that the problem will not be easily remedied. The researchers caution that relying on AI-based image reconstruction techniques to make diagnoses and determine treatment could ultimately do harm to patients. Their results are reported in the Proceedings of the National Academy of Sciences.
"There's been a lot of enthusiasm about AI in medical imaging, and it may well have the potential to revolutionise modern medicine: however, there are potential pitfalls that must not be ignored," said Dr Anders Hansen from Cambridge's Department of Applied Mathematics and Theoretical Physics, who led the research with Dr Ben Adcock from Simon Fraser University. "We've found that AI techniques are highly unstable in medical imaging, so that small changes in the input may result in big changes in the output."
A typical MRI scan can take anywhere between 15 minutes and two hours, depending on the size of the area being scanned and the number of images being taken. The longer the patient spends inside the machine, the higher resolution the final image will be. However, limiting the amount of time patients spend inside the machine is desired, both to reduce the risk to individual patients and to increase the overall number of scans that can be performed.
Using AI techniques to improve the quality of images from MRI scans or other types of medical imaging is an attractive possibility for solving the problem of getting the highest quality image in the smallest amount of time: in theory, AI could take a low-resolution image and make it into a high-resolution version. AI algorithms 'learn' to reconstruct images based on training from previous data, and through this training procedure aim to optimise the quality of the reconstruction. This represents a radical change compared to classical reconstruction techniques that are solely based on mathematical theory without dependency on previous data. In particular, classical techniques do not learn.
[...] The researchers are now focusing on providing the fundamental limits to what can be done with AI techniques. Only when these limits are known will we be able to understand which problems can be solved. "Trial and error-based research would never discover that the alchemists could not make gold: we are in a similar situation with modern AI," said Hansen. "These techniques will never discover their own limitations. Such limitations can only be shown mathematically."
Journal Reference
Vegard Antun, Francesco Renna, Clarice Poon, et al. On instabilities of deep learning in image reconstruction and the potential costs of AI [$], Proceedings of the National Academy of Sciences (DOI: 10.1073/pnas.1907377117)
(Score: 0) by Anonymous Coward on Wednesday May 13 2020, @03:27PM (6 children)
This is like looking at patterns in the clouds. Sure, maybe you'll see boobs but that doesn't make God a pervert.
Seriously. AI is a pattern matching system. The less details you have, the more garbage will come out. AI doesn't make your camera infinite resolution!
https://www.youtube.com/watch?v=I_8ZH1Ggjk0 [youtube.com]
(Score: 0) by Anonymous Coward on Wednesday May 13 2020, @03:45PM
But imagine if it could. The future now. That's why we at $company believe in the power of YOU to take us on a journey. Together.
(Score: 2) by All Your Lawn Are Belong To Us on Wednesday May 13 2020, @04:22PM (3 children)
Yeah, but this isn't just about image interpretation, where pattern matching is important. This is about image construction. This is about trying to take shortcuts in what should be a deterministic process, namely gathering data and sequencing it to become information in the form of images. The end goal is not just making patients more comfortable by reducing bore time (a laudable goal) but also being able to perform more scans per day and increase profitability (which also has its upside to patients but is also questionable). This really isn't a place for a machine to make judgement calls to interpolate data, when by doing so that interpolation actually ends up yielding false data.
This sig for rent.
(Score: 4, Interesting) by JoeMerchant on Wednesday May 13 2020, @04:38PM (2 children)
What matters is the actual results.
Run the proposed system in parallel with the accepted systems. Measure changes in outcomes, decide if the benefits outweigh the costs.
If additional people dying is showing up in the costs column, the system should never get accepted. If people end up having to come back for a second scan 5% of the time, but throughput is increased 50%, that's probably a win overall.
I worked with a 3D scanning system that got direct data off of two intersecting planes, and these were of course the two planes of primary concern, but being a 3D system, there was additional volume off those planes that was still of some interest. We could interpolate cylinders around the line of intersection between the planes and get what would be a reasonable projection of what should be happening there, but not anything that was measured directly. There was a lot of debate about how to present this information, because on the one hand the interpolation made it much easier to visualize when treatment was sufficient and could be stopped before un-necessary collateral damage occurred. On the other hand, interpolation is "made up" information, not measured by the system directly but inferred from the measurements that are being made on the planes. Ultimately, the whole thing was put on the shelf and the "old" system continues to be used now 7 years after the advanced system was developed (different owners, different risk appetites), but in the interim we decided that the interpolated information should be shown but clearly labeled as such to avoid any misconception on the part of the surgeons that they were looking at actual measurements.
🌻🌻 [google.com]
(Score: 2) by All Your Lawn Are Belong To Us on Wednesday May 13 2020, @06:17PM (1 child)
Not quite sure what you mean by, "in parallel." This would be hard to do Level 1 randomized control trials on. Lest you have to say, "Yeah, sorry your father died of that tumor. If we'd only seen it in the initial scans we could have excised it, but that AI extrapolator just wasn't working correctly and your father was in the test group instead of the control group." One can't, AFAICT, take the images and then have the raw images interpreted, have the raw data reconstructed, and then interpreted again as a reconstruction and compare the interpretations, because the essence is that a different (faster) sampling rate is used in the AI generated reconstruction run. One could theoretically do two runs, one after the other at the different rates, but a big confounding variable is patient movement (even minor inadvertent movement) during the scan resulting in different data. Plus you encounter an ethical question about having the patient be in the test significantly longer than normal. (At least there isn't an ionizing radiation concern like a CT or Fluoro would have).
So we end up with the methodology that these researchers used - take the raw data, introduce extremely small data anomalies to the image and then see what the interpretation algorithms do to that known data to see if they distort them significantly in a way that an interpreter might not be able to recognize it.
I'm not saying the study was correct, just that this is what they seem to have done and found. Maybe this does open the door to allow more active testing of the current algorithms. (And that's the other aspect... this certainly reads like these are currently-in-use algorithms that are being tested. So maybe there is need to raise concerns now rather than wait).
This sig for rent.
(Score: 2) by JoeMerchant on Wednesday May 13 2020, @06:48PM
The sequential scans should not be a major concern to the patients - they've already showed up, been prepped, etc. If they happen to be claustrophobic, then they should ethically be excused from the double scan process - there's no need to include claustrophobics in the study, even though they are one of the major beneficiaries of the fast scan process - and differentially the fast scan should be better on squirmy subjects.
If the sequential scans show an acceptable level of interpretation agreement, then there is no reason to believe that that a fast scan alone would be any worse than the fast scan obtained sequentially with a traditional scan.
🌻🌻 [google.com]
(Score: 3, Insightful) by DeathMonkey on Wednesday May 13 2020, @05:29PM
Imagine that, observing things to learn about them!