Project Overview: Using Computer Vision and NLP to Caption X-Rays
Data cleaning, deep learning, and model deployment to replace radiologists.
Automation has been a major driving force of increased efficiency, reliability, and speed across multiple industries, ranging from banking to transportation to agriculture. In this project, we investigate the potential of deep learning models to automate the process of medical image reporting, looking specifically at chest X-ray images.
Developing a deep learning model to generate/support the reporting of findings and impressions from X-ray images would be a highly valuable development since it takes radiologists a significant amount of time to carry out this process for a large number of patients. Depending on the level of correctness achieved by the model, it may also be able to reduce human error, which is especially costly in the medical field.
In this series, we dive deep into the Chest X-Rays Indiana University dataset hosted on Kaggle. Our process has been broken down into the following topics:
- Part 1: Cleaning and Pre-processing X-Ray Data
- Part 2: Exploring and Engineering New X-Ray Data Features
- Part 3: Creating a Caption Generating Model Using CNNs and RNNs
- Part 4: Deploying the Model to Serve X-Ray Diagnosis in Production
- Part 5: Interpreting Machine-Predicted X-Ray Captions and Concluding Remarks
The goal of this project is to measure the similarity of machine-predicted captions to actual captions provided by doctors.
The code is hosted and useable at this GitHub repository.
