How are AutoML techniques changing the face of machine learning?
For all the cheap computer owners out there, NAS and AutoML are like dreams come true. Personally, I own a Macbook Pro 2017 and being a student on a budget, getting deep learning models to actually run has been a nightmare. My laptop has run continuous days even weeks training YOLOv4, YOLOv3, Inception and SSD models. NAS changes everything! With the work going on in and around the Lottery Ticket Hypothesis, I think researchers are taking AI to the next paradigm. For all the Kagglers out there, it must be tough to compete against participants running complex models on multi-GPUs and ironically having a higher chance of winning.
Before I delved into why I am talking about this let’s talk about the what. A recent study proved that very sparse Neural Networks whose weights have been initialised as those of the extremely deep networks can achieve equitable levels of accuracy without much lower computational requirements and guess what, lower inference and training times? What does this mean? We CPU users stand a chance. The researchers tested their hypothesis, which is now commonly known as the Lottery Ticket Hypothesis, against the MNIST and CIFAR-10 datasets and surprisingly achieved equal level of accuracies with networks that were 10–20% of the original extremely vast feed forward neural networks with several fully connected layers. Their research dates back to what many players in Deep Learning have been talking about for a long time, but the were able to prove it! We all know that irrelevant weights add much bulk to our models and for CPU users some crucial training time that might just save their laptops. The argument that the researchers put forward was that the bulk of the job is done by a subsection of our vast deep networks, and the rest is redundant thus we can replicate performances of very deep networks with extremely sparse features. It is safe to say that this research has true potential and we are reaching a time in tech exploration wherein having less is truly better!
Another thing, that I would like to talk about today is Neural Architecture Search(NAS). It is impossible to create all permutations and combinations of Neural Networks, but what if there existed a technique wherein a network can be made to best fulfil your tasks well using another branch in AI, this has also been made possible using Neural Architecture Search techniques. To give you a high level overview NAS uses techniques of Reinforcements Learning and Gradient Optimization and adapts its systems of states and rewards to create or choose an optimal architecture and the appropriate parameters. Now, NAS were built in pursuit of bringing ML to the masses and essentially allow people to do research and leverage this complicated technology with limited domain knowledge. But is NAS overpowering everything? Not as of now, NAS or the broader field Automated Machine Learning has a long way to go, it has its own set of drawbacks and hasn’t proven itself worthy enough to be used everywhere. Anyway, NAS is really cool and I believe it is a futuristic technique that will definitely make AI available for all.
I think by this time it must be clear why I am harping upon these concepts. Not only is this a great step towards ‘Deep Learning for All’ but also an avenue for better exploration in edge and ‘lite’ devices. Sparse neural networks have a much better chance of running accurately and faster than those converted Deep Networks. So imagine in a few short years, all that we see in our laptops with GPUs can be available on the tip of our fingers with our phones!
NAS and AutoML were a great boost for me for my upcoming project. My friend and I decided to create a trash classifier for the sustainable consumer. We discovered how safe trash disposal especially in India could be a leap forward in terms sustainability, but there is just one problem: information. Even I didn’t the specifications to trash disposal that are out there and the project introduced a great learning curve for me. Instead of taking the relatively tumultuous route of networks from scratch or transfer learning, AutoML came to our rescue. Overall we used softwares like CreateML, Teachable Machine and AutoKeras to make our classifiers. It took only a few hours this time for our models to reach impressive accuracy levels. We created models to classify trash into our defined categories and informed consumers about what they are going to dispose and how to dispose it. Behind the scenes our easy-to-use NAS backed models can figure out basic categories like paper, trash, metal, plastic etc. but also go one step further to tell you to whether to compost it, recycle it or trash it. For good measure we even included a plastic classifier which was broadly defined into 7 categories of plastic, each of which have different disposal methods. We used these models and converted them into edge usable formats and published an iOS app. We are constantly looking to improvise the app so go search ‘Garbify’ on the app store and drop in a review!
https://apps.apple.com/in/app/garbify/id1522674990 — App Link. It is published under another account because I didn’t have an apple account :(
Our next steps are to create a custom analytics backend and also include an e-waste recogniser addition to our models.
That’s it from me! Let me know what you think about the app!
Lottery Ticket Hypothesis Research Paper — https://arxiv.org/abs/1803.03635
