Radiologists review thousands of images a day. The hope is that artificial intelligence (AI) applications will become useful soon to verify diagnoses, prioritize queued images and even to offer a level of detection and measurement that aren't feasible for humans. One of the latest efforts on this front is by researchers at the University of California at San Francisco (UCSF) and the University of California at Berkeley.
They developed a deep learning system that did better than two out of four expert radiologists in detecting tiny brain bleeds on CT scans. The study results were published in the Oct. 21 issue of the Proceedings of the National Academy of Sciences. The algorithm is being further tested in the ongoing traumatic brain injury study TRACK-TBI that is a public-private endeavor and the largest trial of its kind with more than 3,800 patients at 18 trauma centers including UCSF.
"My research is in traumatic brain injury, so I read thousands of scans for a multicenter study through UCSF. We'd been trying to test the algorithm repeatedly over a couple of years," Esther Yuh, an associate professor of radiology at UCSF and co-corresponding author of the study, told BioWorld MedTech.
"There was a point one day when we ran a test and I looked at the results; I was really shocked because it had taken a jump which was pretty abrupt. I actually couldn't believe my eyes when I was looking through it and looking at the answers," she continued. "I realized that it was actually something I could trust and at that point it reached a level where it really knocked my socks off. Before that time, there were little imperfections here and there would be a mistake in every two or three exams."
The algorithm is based on a specific kind of deep learning technology known as a fully convolutional neural network; it was trained using 4,396 head CT scans. Then, using 200 randomly selected head CT scans, the algorithm was tested against evaluations by four expert radiologists.
The algorithm demonstrated the highest accuracy found to date for this clinical application: an area under the curve of 99.1. It exceeded the performance of two out of the four radiologists.
"We took the approach of marking out every abnormality – that's why we had much, much better data," said Jitendra Malik, the Arthur Chick Professor of Electrical Engineering and Computer Sciences at Berkeley and a co-corresponding author of the study. "Then we made the best use possible of that data. That's how we achieved success."
He added, "Given the large number of people who suffer from traumatic brain injury every day and are rushed to the emergency department, this has very big clinical importance. That convinced me to work on this problem.
Yuh expects the algorithm will continue to improve as it's used to analyze more CT scans, since it's continuously learning. The system was particularly good at finding very small brain bleeds, and it also looks at all the full scans. Radiologists often look on the image where they expect to find a problem and examine only a handful of the best images, rather than all the 30 CT scans that are typically produced.
She anticipates that the algorithm could be useful at first simply to verify the diagnosis of an individual radiologists, but eventually it might enable a pre-screen that could provide a radiologist with a report to consider in advance of viewing CT scans or even to prioritize which images will be analyzed first. But for those more advanced applications, the system would need to be flawless to avoid burying important scans at the bottom of the imaging queue.
Summed up the paper, "We demonstrate an end-to-end network that performs joint classification and segmentation with examination-level classification comparable to experts, in addition to robust localization of abnormalities, including some that are missed by radiologists, both of which are critically important elements for this application."
Improving TBI biomarkers
The algorithm is currently part of the ongoing TRACK-TBI study, where it's being used to assess CT scans. Imaging is being used alongside blood biomarker data to identify TBI patients. Yuh pointed out that while this algorithm increases the detection of the smallest events, blood biomarker tests are getting increasingly sensitive.
She was an author on an August paper published in The Lancet Neurology that showed that a blood test can identify TBI events that are undetectable via imaging. Specifically, the researchers used the Abbott Laboratories' hand-held Alinity device to detect a blood biomarker, which was glial fibrillary acidic protein. This was also part of the TRACK-TBI study.
Yuh said the academic researchers don't have the bandwidth or the capacity to push the AI algorithm to detect brain bleeds much further without a partnership. She's hopeful that the technology can transition to a clear pathway toward regulators and commercialization, particularly since she expects that this sort of technology could quickly make a big impact for patients and radiologists.
"This struck me as something that is applicable clinically in the near future and could really vastly change the way things are done for the better in many ways. There don't seem to be many drawbacks," she said. "The advantages of this seem to me to be completely across the board, in that it helps patients and it makes radiologist more efficient. It allows you to gather other types of information that people cannot do."
"Radiologists don't have the time to look in detail at, for example, measuring the size of the things. That's just extremely time consuming and something that we can't do on a practical basis. There's no way you can actually get through the stack of images if you want to do something like that, whereas the computer can do that. It's basically less than a second to process these," concluded Yuh. "So, if you can achieve better accuracy, it's far faster, and the patient gets faster results. It's at least as accurate, and hopefully can actually be better than the average radiologist and probably better across the board."