Meta Learning — Learning To Learn
As the AI technology progresses we are starting to see neural networks able to multi-task. We have seen the use case of transfer learning. It gives a jump start to training new similar task and subtask. Recently the AI trends have moved towards Meta-Learning whereby instead of focusing on training the neural network on data distribution, the neural networks are trained on tasks. We, humans, are good at using our prior experience to work on a previously unseen task. We can learn to play Atari games by observing faster than a Reinforcement Learning algorithm does. We are now moving into a place where we are solving meta-heuristics to the problem thereby making it generalize better.
What we have seen so far this year
Neural Network That Build Neural Network Architectures
NAS — Stands for Neural Architectural Search to find a neural network that does well on tasks. NASNet was a paper that was released this year. NASNet used RNN to create a neural network by working on the metaheuristics of CNN combination. It significantly reduced the training time. NASNet created a scalable algorithm that can be used in multiple image recognition and object detection tasks.
It achieved state of the art performance on the Imagenet dataset for a top-1 prediction and did well in object detection task setting a map score of 43.2 when added as a base network to faster-RCNN.
It also showed few promising characteristics
** State of the art performance was achieved with smaller parameters.
** It formed Dense Connection(skip connection) on its own thereby opening up the possibility of a better network.
** Parallel architecture in CNN was explored(Normal cell contain 5 CNN combination same with reduced cell).
** It’s a Scalable Architecture.
There are many techniques in the same space that worked were reinforcement learning, evolutionary strategies. NASNet was used in AutoML by google.
Reptile
This paper showed how Neural Network could learn to multiple tasks instead of learning a data distribution of a single task. It’s learning to find the initial weights that would generalize faster to other tasks. It’s learning the loss function of the task.
There are a few methods that were used prior such as MAML that used second-order gradient derivative to find weight initialization. Then they revised the MAML model with first-order gradient derivative to find weight initialization that would generalize to task quickly.
OpenAI came up with this Reptile algorithm that shows if gradients are updated after t > 1 then it produces an equivalent result of that of MAML in the optimizer.
Learning to see
This is a big paper in the Computer Vision space developed by deep minds. It’s so intriguing that we humans have a prior understanding of a scene we inherently know a table has 4 legs even if posterior legs aren’t visible from our vision.
They created a Representative and Generative network. The input to the network is a combination of 2D image and viewpoint angles. The representative network converts it to an encoding representation of the scene. The Generative network gets that representation and a new viewpoint to generate scene from that perspective.
The beauty of the GAN is that they are able to capture scene prospective, lighting changes, objects in the scene and their prospective to the scene, geometry, occulation and many other intricacies. They were able to create a 3d rendering of the scene from few 2d viewpoint images. 3D rendering captured subtle changes in lighting, object position, overall scene, even occulation, and geometry of objects in the scene and their relation to the scene.
Learning To Drive A Car In 20 Minutes
This paper showed the self-driving car community that by reshaping reward and MDP of the problem faster training could be made with the monocular camera and a driver. They were able to drive a car from a single camera feed. They took their intuition from humans, we learn to ride the bicycle with the help and within a few rides, we are able to learn some basics into riding a bicycle. Then they reshaped reward to distance traveled without human control. This helped them to train the Actor-critic network faster. They also showed encoding convolutional maps created better results.
Neural Combinatorial Optimization
Traveling salesman & vehicular routing problems are np-hard problems to solve for the computer science community. It was solved by designing heuristics using Genetic Algorithm. Especially after RNN models came about these Combinatorial Optimization problems could effectively be solved by using reinforcement learning and RNN.
The cities or customers are represented in a sequence and feed to RNN pointer network. Critic network is used to find optimal distance estimates or time estimates to make the training faster for the actor-network.
This opens door to solving np-hard problem that have difficulty in creating Heuristics.
Evolutionary Strategies with reinforcement learning
Complex environments are hard to solve due to sparse reward in reinforcement learning. There are problems were functions are non-differentiable. The evolutionary algorithm can step in to search the space for the optimal solution and feedback from reinforcement learning could be used to optimize the algorithm faster.
This year has been so good for the AI community. We’re starting to see multi-task solving neural networks, learning to think and see like humans, GAN’s are used in many problems. As we progress we will find more application of meta-learning in many tasks like Imagenet, Object detection, NLP, optimization etc.
Thanks for reading through. Hope that made up your time. A clap would send this post to many other people in the AI community. New ideas are always welcomed in the comments. Follow us on Social to follow this series on Meta-Learning → McGroce.
