I’m sure you have seen this article in Nature, http://www.nature.com/news/there-is-a-blind-spot-in-ai-research-1.20805, which makes much the same argument. The article mentions Hinton’s team reversing the flow of information through a CNN to try and figure out what the input might look like, given the output, which is an interesting step in this direction.
As a victim of bad surgery, multiple incorrect diagnoses, and as a fanatical cyclist, I would already trust a machines over a human when it comes to medicinal diagnosis or driving a car, for example, but I find my friends who don’t work anywhere near AI have a hard time accepting this. It is of paramount importance that we learn to explain how computers “learn” to the wider public for these systems to gain wider acceptance. Naturally, such understanding will help scientists working in the field also.