Guided Transfer Learning: How to use ‘the power of scouts’ to boost machine learning performance

An exclusive sneak-peek at a revolutionary new method for training neural networks

In this newly proposed technique, small ‘scout models’ are sent to navigate the problem landscape and ‘report back’ to the main model. Photo by @ansgarscheffold on Unsplash.

My good friend and humble genius Dr Danko Nikolić recently shared an unpublished paper with me, thinking I might be interested. Was I ever. Reading it made me feel like I was witnessing a historic moment before anyone else did, and I was immediately bursting to share. Thankfully, Danko agreed. So here’s my translation into day-to-day language of a method I think could revolutionise the training of deep neural networks. It’s not even out on arXiv yet (update: now it is!), but NASA are already using it. So once it does blow up, remember: you heard it here first. 😉

Let’s Start With The Problem

I’m sure you know it: machine learning, especially with deep neural networks, requires a frankly ludicrous amount of data, compute power, and model parameters. This makes them inaccessible to all but the wealthiest of companies and research institutions, and thus concentrates — into the hands of a small few — the power to develop AI technologies that will shape our technological future. Not cool.

Why The Problem Exists