A new AI algorithm was better able to distinguish COVID-19 induced pneumonia from non-COVID-19 pneumonia in chest X-rays than three thoracic radiologists
AI algorithm distinguishes COVID-19 pneumonia in chest X-ray from other abnormalities
September 30, 2020
by John R. Fischer
, Senior Reporter
Radiologists at Henry Ford Health System and the University of Wisconsin-Madison have developed an AI algorithm that surpassed experienced thoracic radiologists in distinguishing COVID-19 pneumonia from other causes of abnormalities on chest X-rays.
Called CV19-NET, the solution is expected by its creators to help radiologists differentiate CXRs with COVID-19 pneumonia symptoms from those without. Its use may also generate greater reliance on CXRs which, while recommended by major medical societies for diagnosing the disease, have low sensitivity and specificity that hinders their use as a diagnostic tool.
"Algorithms like this will help healthcare providers identify patients at high risk for COVID-19 pneumonia immediately after a chest X-ray is taken, even before the pictures are sent back to a reading room for the radiologist to review," author Dr. Scott Reeder, professor of radiology and medical physics at the University of Wisconsin-Madison, told HCB News. "Such algorithms will help radiologists to triage, i.e., prioritize their workflow by identifying those cases with a high probability COVID-19 and read those chest X-rays first."
Reverse transcriptase polymerase chain reaction (RT-PCR) is the current reference standard approach for identifying patients with the virus. While CT has been widely used in China and a few other countries to do the same, concerns over contamination of CT imaging facilities and exposure to healthcare workers have led healthcare professional organizations to dissuade providers from relying on them for making COVID-19 diagnoses.
The researchers used 2,654 CXRs from 1,962 patients with non-COVID-19 pneumonia, and 2,582 from 1,053 with RT-PCR-confirmed COVID-19 to train and validate their solution, CV19-NET. They then used 2,646 CXRs from 1,186 patients with non-COVID-19 pneumonia and 3,223 CXRs from 1007 patients with RT-PCR-confirmed COVID-19 to test it, resulting in 5,869 CXR images from 2,193 patients within the test dataset.
All CXRs were split into training + validation and test datasets, and all were randomly selected from Carestream Health, GE Healthcare, Konica Minolta and Agfa to ensure the algorithm could be applied broadly to scans taken at different facilities which use imaging systems from different vendors. Area under the receiver operating characteristic curve (AUC), sensitivity and specificity were calculated to characterize diagnostic performance and tested over the entire test cohort of 5,869 CXRs.
CV19-NET surpassed three experienced thoracic radiologists in distinguishing pneumonia associated with COVID-19 from other types, achieving an area under the curve of 0.92 on the test dataset of 5,869 CXRs from the 2,193 patients acquired from multiple hospitals and multiple vendors. When presented with a set of 500 randomly selected test CXRs, the AI algorithm achieved an AUC of 0.94, compared to an AUC of 0.85 from the three.
Many radiologists are only just seeing COVID-19-induced pneumonia cases for the first time, and require more cases to read in order to learn the common and unique imaging features associated with the disease. This, along with similar algorithms in development, is expected to help bridge this learning gap.
"Since the start of the pandemic, many AI researchers, including medical physicists, computer scientists, biomedical engineers, and healthcare industry researchers have been working on AI algorithms like this," Guang-Hong Chen, Ph.D., professor of medical physics and radiology at University of Wisconsin-Madison, told HCB News. "In the next year or so, there will be more algorithms available to differentiate symptoms of COVID-19 from non-COVID-19 conditions. The deployment of the algorithms like this may help healthcare providers quickly identify those patients with high risk of COVID-19 from other patients without COVID-19, despite similar clinical symptoms such as fever or cough."
The findings were published in Radiology.