You are here


Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation

Primary tabs

The National Institutes of Health - Clinical Center (NIH-CC) seeks to license and/or co-develop methods of reading chest x-rays using a deep learning models to detect a disease and describe its contents.
NIH Reference Number
Product Type
  • Chest X-Rays, Deep Learning Model, Disease Mining from Images, Computer Assisted Diagnostics, Radiology, Convolutional Neural Networks, CNN, Recurrent Neural Networks, RNN, Shin
Collaboration Opportunity
This invention is available for licensing and co-development.
Description of Technology

Medical image datasets are an important clinical resource. Effectively referencing patient images against similar related images and case histories can inform and produce better treatment outcomes. Labeling and identifying disease features and relations between images within a large image database has not been a task capable of automation. Rather, it is a task that must be performed by highly trained clinicians who can identify and label the medically meaningful image features. The time constraints on clinicians versus the size of these data sets greatly limits the ability to analyze, annotate, and interrelate images in such a database, so these efforts are not routinely completed for large datasets.
Researchers from the National Institutes of Health - Clinical Center (NIH-CC) have developed a technology that automatically detects disease features and applies labels & annotation of clinical findings in a chest x-rays’ images using deep learning methods. A training set of image annotations is mined for disease names to train convolutional neural networks (CNNs). Recurrent neural networks (RNNs) are then trained to describe the context of a detected disease’s features, building on top of the deep CNN features. Feedback from the trained pair of CNN/RNN can then be used to infer the joint image/text contexts for composite image labeling (location, severity, and the affected organs). The capability for automation of methods to search and extract meaningful medical information from images might transform the usefulness of medical image databases, making them searchable and usable for detection and diagnostic applications in cloud-based services. 

Potential Commercial Applications
  • Computer assisted diagnostic
  • Medical image data set mining 
  • Cloud-based services 
Competitive Advantages
  • Automated diseases detection
  • Ability to automatically annotate contexts 
Development Stage

Shin H, et al. Learning to read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation   [Article]

Patent Status
  • Research Material: NIH will not pursue patent prosecution for this technology
Therapeutic Area
Sunday, August 16, 2020