Automated Caries Detection on Bitewing Radiographs Using Deep CNNs

Research has established that a large percentage of dental caries escape identification in routine dental examinations, even when such examinations include dental x-rays. Certain types of caries, like occlusal caries, appear to be easier to find through a normal clinical examination or x-ray review, whereas diagnosing other types of caries, such as caries below the surface of the tooth, interproximal caries, and root caries, is often not as reliable.

At ParallelDots, Inc. we took the challenge to find these dental anomalies with human-level accuracy and build a reliable diagnostic tool for the dentists. In this blog post, we will discuss the recent study we did for our automated caries detection system, putting our system against three practicing dentists from North American clinics. We found that our system had a higher agreement (F Score) with clinically verified ground truth than all three individually (the difference between system’s F Score and average F-Score of the dentists is over 17%). Our system has higher sensitivity with respect to Dentists individually and hence can be used as a tool to ease the work of dentists by suggesting them possible caries they can then verify and treat. A breakdown of metrics in our test is given in the table below. Please note that the metrics Recall and Precision are names Machine Learning community generally uses for Sensitivity and Positive Predictive Value.

Our paper detailing the experiments performed has been accepted at the NIPS 2017 Workshop on Machine Learning for Health being held on the theme “What Parts of Healthcare are Ripe for Disruption by Machine Learning Right Now?”. NIPS (Neural Information Processing Systems) is among the topmost Machine Learning conferences globally and has two tracks of papers and multiple focused workshops.

How does the automated caries detection technology work?

The first step is to gather a dataset to train AI algorithm on. We obtained an IRB approval and collected data from multiple clinics across the US, where dentists marked clinically verified cavities on their XRay archives. 3000 radiographs with cavities annotated were collected as a part of this exercise. The data is then anonymized as soon as the clinician uploads it to our HIPAA compliant server and the dataset thus is ready to be used by our Data Science team.

The next step for the Data Science team is to decide:
a. how to model this as a machine learning problem
b. How to calculate the accuracy of the machine learning system thus created.
Generally, the processes a and b go hand in hand (for example you would measure accuracy for something you would model as a pixelwise classification task using dice coefficient, for an object detection task as Intersection over Union or for a classification task as accuracy). In this case, however, the problem arises due to the following complications:

1. Dental Cavities are irregular shaped, thus there is no way to mark them exactly by the dentists. At best we can get approximate polygon annotations for dental caries. Although modeling the cavity detection problem as Dense Classification task is pretty obvious, a dice score will not be a fair measure of accuracy on such approximate annotations.
2. While average IoU score might be a good measure it is hard to visualize how effective the model is just by this number in the real world. Hence we put a more practical criterion for the model to be evaluated. We evaluate the model as if it were a search engine to find cavities. A search engine is evaluated based on measures of precision, recall and F Score. Precision denotes the number of correct searches among total results returned by the cavity search. Out of the total cavities present, the number of cavities that the system can detect is called Recall.

A radiograph containing caries and the output processed by our algorithm is shown below.

With this research, we are aiming to assist dentists in analyzing and detecting dental problems. The automated caries detection system can provide a solid assistance to dentists and make their work easier, faster and more accurate so that they can effectively concentrate on rectifying the dental problems much faster.

At ParallalDots, we are working on challenging problems in Healthcare, NLP and Image Classification. Please watch this space for more updates on the research work at ParallelDots.

ParallelDots AI APIs, is a Deep Learning powered web service by ParallelDots Inc, that can comprehend a huge amount of unstructured text and visual content to empower your products. You can check out some of our text analysis APIs and reach out to us by filling this form here or write to us at apis@paralleldots.com.