Integrating medical imaging and cancer biology with deep neural networks

Integrating medical imaging and cancer biology with deep neural networks
Neural network framework. Credit: Smedley, Aberle, and Hsu

Despite our remarkable advances in medicine and healthcare, the cure to cancer continues to elude us. On the bright side, we have made considerable progress in detecting several cancers in earlier stages, allowing doctors to provide treatments that increase long-term survival. The credit for this is due to "integrated diagnosis," an approach to patient care that combines molecular information and medical imaging data to diagnose the cancer type and, eventually, predict treatment outcomes.

There are, however, several intricacies involved. The correlation of molecular patterns, such as and mutation, with image features (e.g., how a tumor appears in a CT scan), is commonly referred to as "radiogenomics." This field is limited by its frequent use of high-dimensional data, wherein the number of features exceeds that of observations. Radiogenomics is also plagued by several simplifying model assumptions and a lack of validation datasets. While machine learning techniques such as deep neural networks can alleviate this situation by providing accurate predictions of image features from gene expression patterns, there arises a new problem: We do not know what the model has learned.

"The ability to interrogate the model is critical to understanding and validating the learned radiogenomic associations," explains William Hsu, associate professor of radiological sciences at the University of California, Los Angeles, and director of the Integrated Diagnostics Shared Resource. Hsu's lab works on problems related to data integration, machine learning, and imaging informatics. In an earlier study, Hsu and his colleagues used a method of interpreting a called "gene masking" to interrogate trained neural networks to understand learned associations between genes and imaging phenotypes. They demonstrated that the radiogenomic associations discovered by their model were consistent with prior knowledge. However, they only used a single for brain tumor in their previous study, which means the generalizability of their approach remained to be determined.

Integrating medical imaging and cancer biology with deep neural networks
In a new study, researchers tested the ability of their deep neural network model to predict radiogenomic features against that of other classifiers within the same training dataset, and found that its overall performance was better. Credit: Smedley, Aberle, and Hsu

Against this backdrop, Hsu and his colleagues, Nova Smedley, former graduate student and lead author, and Denise Aberle, a thoracic radiologist, have carried out a study investigating whether can represent associations between gene expression, histology (microscopic features of biological tissues), and CT-derived image features. They found that the could not only reproduce previously reported associations but also identify new ones. The results of this study are published in the Journal of Medical Imaging.

The researchers used a dataset of 262 patients to train their neural networks to predict 101 features from a massive collection of 21,766 gene expressions. They then tested its predictive ability on an independent dataset of 89 patients, while pitting its ability against that of other models within the training dataset. Finally, they applied gene masking to determine the learned associations between subsets of and the type of lung cancer.

They found that the overall performance of neural networks at representing these datasets was better than the other models and generalizable to datasets from another population. Additionally, the results of gene masking suggested that the prediction of each imaging feature was related to a unique gene expression profile governed by biological processes.

The researchers are encouraged by their findings. "While radiogenomic associations have previously been shown to accurately risk stratify patients, we are excited by the prospect that our can better identify and understand the significance of these associations. We hope this approach increases the radiologist's confidence in assessing the type of lung cancer seen on a CT scan. This information would be highly beneficial in informing individualized treatment planning," observes Hsu.

More information: Nova F. Smedley et al, Using deep neural networks and interpretability methods to identify gene expression patterns that predict radiomic features and histology in non-small cell lung cancer, Journal of Medical Imaging (2021). DOI: 10.1117/1.JMI.8.3.031906

Provided by SPIE
Citation: Integrating medical imaging and cancer biology with deep neural networks (2021, May 10) retrieved 24 September 2023 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Deep learning may help doctors choose better lung cancer treatments


Feedback to editors