Auto-Keras and Auto-ML — Weekend of a Data Scientist

Alexander Osipenko
Aug 31, 2018 · 3 min read

Weekend of a Data Scientist is series of articles with some cool stuff I care about. Idea is to spend weekend by learning something new, reading and coding.

In 2017, Google released a blog post and paper that created a lot of hype in the industry. The thing is that they successfully developed the system that supposedly can take your data and come up with the best neural network architecture for that data. It was called auto-ML, and news articles were saying that it can take a job from Data Scientist or at least will dramatically change the way we work.

So several weeks ago Auto-ML was finally released as a product (beta version) and it cost 20$ per hour!

Now. Even that Google’s auto-ml uses state of the art models for picture recognition, we should keep in mind, that they were not the first who come up with this idea. There are several open-source libraries, that on a high level can do kind of the same thing, e.g. take your data and come up with best and already tuned models, for example, TPOT and auto-sklearn. But frankly speaking, they not exactly suitable for deep neural nets and they don’t use the same approach as Google’s Auto-ML — Efficient Neural Architecture Search (ENAS)

So what is Efficient Neural Architecture Search?

ENAS is an approach for the automatic model design. In ENAS, a controller learns to discover neural network architectures by searching for an optimal subgraph within a large computational graph. The controller is trained with a policy gradient to select a subgraph that maximises the expected reward on the validation set. Meanwhile, the model corresponding to the selected subgraph is trained to minimise a canonical cross entropy loss. It’s 1000x less expensive than the standard Neural Architecture Search due to parameter sharing between child models, so there is no need to train the model from the ground on each iteration.

animation from ENAS PyTorch implementation

Open-source ENAS solutions

If you don’t want to pay 20$ per hour, you can try different open-source libraries that use the ENAS approach.

  1. Auto-Keras — https://autokeras.com

2. ENAS in PyTorch — https://github.com/carpedm20/ENAS-pytorch

3. ENAS in TensorFlow— https://github.com/melodyguan/enas

They all pretty simple to run, for example in Auto-Keras you will need 4 lines of code to build an image classifier:

In conclusion

The purpose of this systems is to allow people who are not experts in Machine Learning to build and train complicated deep neural nets in a fairly simple way and use them for applied purposes without wasting time on the tedious process of model building and tuning. So open-source solution, in my opinion, is a very good place to start with ENAS!

References:

  1. Efficient Neural Architecture Search via Parameter Sharing
  2. Learning Transferable Architectures for Scalable Image Recognition
  3. AutoML for large-scale image classification and object detection

Did you try Auto-ML or Auto-Keras?


Cindicator

Cindicator is a fintech company that enables effective asset management through predictive analytics based on Hybrid Intelligence. Here we share our news & views on token economy, smart money, Black Swans, data analysis, AI, Machine Learning, and other topics.

Alexander Osipenko

Written by

Data Scientist passioned in Deep Learning and Time Series analysis

Cindicator

Cindicator is a fintech company that enables effective asset management through predictive analytics based on Hybrid Intelligence. Here we share our news & views on token economy, smart money, Black Swans, data analysis, AI, Machine Learning, and other topics.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade