Andrew Ng, Lior Pachter & Gary Marcus Twitter Joust on AI Radiology

“It’s quite obvious that we should stop training radiologists,” said deep learning pioneer Geoffrey Hinton in late 2016. The remark ruffled more than a few feathers in medicine, as it was widely regarded as a eulogy for radiology professionals.

This week Andrew Ng tweeted that his Stanford University team had developed a model that is “comparable to the best radiologist performance in detecting abnormalities on finger and wrist studies. However, model performance is lower than best radiologist performance in detecting abnormalities on elbow, forearm, hand, humerus, and shoulder.”

Ng’s team at Stanford got the results with a 169-layer convolutional neural network (CNN) trained on the Stanford lab’s new bone X-ray dataset MURA, for “musculoskeletal radiographs” — said to be the world’s largest public radiographic image dataset, with 40,561 labeled images.

In response to Ng’s tweet, Caltech computational biologist Lior Pachter pointed out the contradiction with an earlier Ng Tweet announcing the success of Stanford ML’s ChexNet model with the bold statement: “should radiologists be worried about their jobs? Breaking news: We can now diagnose pneumonia from chest X-rays better than radiologists.”

Eric Topol, a physician-scientist challenged Ng’s claim based on its sample size, “the arXiv preprint CheXNet suggests, at best, matched 4 academic radiologists. One was barely outperformed, which affected the average. Are 4 radiologists representative of the profession?”

Although Stanford’s earlier CheXNet model had garnered some criticisms, these focused less on the model’s actual performance and more on the deep learning community’s suggestions that radiologists would soon be jobless. Pachter recalled a comment from his former student Harold Pimentel, now a postdoc at Stanford, who said: “No, [the models will not replace jobs]. Also, let’s agree to be a bit more responsible with our ledes. Thanks.”

After Pachter’s Twitter thread had harvested more than 1,500 likes, NYU Professor of Psychology and Neural Science Gary Marcus backed up Pachter et al. and even took a swipe at Ng: “overhyping is perhaps getting to be a habit for Andrew Ng, in radiology, cars, and AI more generally…”

Discussion in the Twitterverse expanded to include IBM Watson’s recent setbacks with AI in healthcare, while some even referred to Theano’s fraudulent claims.

Ok, let’s get back to Geoffrey Hinton, whose belief that radiologists would soon be out of work started all this. In last year’s New York Times feature story A.I. VERSUS M.D., Hinton doubled down: “I think that if you work as a radiologist you are like Wile E. Coyote in the cartoon, you are already over the edge of the cliff, but you haven’t yet looked down… It’s just completely obvious that in five years deep learning is going to do better than radiologists. It might be ten years. I said this at a hospital. It did not go down too well.”

* * *

Journalist: Meghan Han | Editor: Michael Sarazen

* * *

Subscribe here to get insightful tech news, reviews and analysis!

* * *

The ATEC Artificial Intelligence Competition is a fintech algorithm competition hosted by Ant Financial for top global data algorithm developers. It focuses on highly-important industry-level fintech issues, provides a prize pool worth millions. Register now!