What’s missing in Deep Neural Network?
Proliferation usage of Deep Neural is still limited by 3 major bottleneck
- Absurd compute and memory requirement
- Huge amount of labeled dataset requirement
- You don’t trust this model because you don’t know how they works.
AiOTA Labs research is focused on overcoming all these bottleneck. We have already shown that we have the ability to compress any DNN in the wild by factor ranging from 3X to as high as 20X(memory footprint, compute complexity, power, speeding the inference time)
In today’s blog (may be it will be a series of blog), I will be mainly discussing about #2 and #3 bottlenecks.
OK, Lets start.
Despite the huge success and popularity of Deep Neural Network(DNN) there are plenty of skepticism in its usage specifically by safety/mission critical applications such as autonomous vehicles(AV), aviation industries, financial technology institutes(fintech) or any other industries where the usages of the DNN directly impact human life or financial related decision.
But the question is Why? Why there is so much reluctant in using these wonderful machine which has demonstrated to solve complex task which human can’t do. The answer is you simply don’t trust its output because you are not sure on which basis it has generated those outputs. In other words DNN is still a big mystery to us on how does it solves these complex problem but we know that it solves!! And when you are not in control of these machines you don’t bet on it. DNN has to be answerable to humans on why and how it arrives to certain decision. Unfortunately these reasoning is absent in deterministic model based decision making such as DNN. As an example in the case of classification, it just provide you a probability of certain class relative to other class. What if it has not seen any other out-of-class object on which it was trained. The same is true in case of DNN based regression task. How they will behave if out-of-class object is shown to it. Will it tell to the user that I’m not sure about this particular class and pass on the decision making ability to human?
Present deterministic DNN frameworks lacks the ability of reasoning and that is the reason Auto industry, aviation industry or banking sector are too much skeptic in usage of this machine.
Recent event which raises more doubt on DNN are the infamous AV accidents where the perception module couldn’t capture some obstacles ( white truck, cyclist) and leads to some tragic human life loss, facebook chatbots developed their own language or a classifier unfortunately classified American-African as Gorillas raising serious racial abuses. If this is not enough, here comes the entry of adversarial attacks on DNN which completely fools the DNN by even just one pixel manipulation. It can start over-confidently under such attacks, classify a dog with zebra or it can misinterpret a stop sign with go sign with some pixel manipulation. You don’t even need to intentionally hack the system with pixel manipulation since there are high chances that camera sensors pixels can degrade over the year leading to unintended adversarial attacks.
So a best DNN based system should not be just a deterministic probabilistic output(like the present one) but should also have a confidence metric associated with each probabilistic output. Now the DNN can use the combination of these two outputs to take more confident or less confident decision compared to a present way of taking only probabilistic based decision. With this it can also give a reason on every decision it make like if user queries that why it took a particular decision it should say for example in case of AV, ‘that I have slowed down in a cross-road because I am not sure that car on the other side will turn right, left or go straight. This type of reasoning will gain trust and confidence of the user on DNN’.
So until or unless DNN based application guarantees the safety of the human life and start giving reasoning on the decisions it has made, I personally has a little hope on proliferation usage of DNN.
Now lets spend sometime in bottleneck #2 i.e labeled training dataset. A DNN will best work if you trained with many many many …. labeled dataset i.e you convert a stochastic world to a deterministic world. But creating labeled dataset which converts a stochastic world to deterministic world is challenging and costly. You are not sure that you have captured the entire world in your dataset. How about a DNN which tells to the user that which data points are most interesting and tell the oracle to label only those data points. WoW, that would be wonderful if DNN is telling to oracle to show only those dataset which are most meaningful on its own world.
Can there be some magic which can club #2 and #3 in single model definition. This is where AiOTA Labs researcher are focused at. Guaranteed safety while reasoning out why it has taken a certain decision and also simultaneously tell to the oracle which are the most interesting data points on training data set which makes a DNN a near-perfect machine.
My next series of blog post will be based on revealing some of the secrets on which a DNN can do all these impossible things.
Till than have healthy wonderful life ahead!!
