Interview with Ian Goodfellow, Research Scientist at OpenAI

July 2016

Ian Goodfellow is a top machine learning contributor and research scientist at OpenAI. Not only did he invent Generative Adversarial Networks (GANs), max-out networks, multi-prediction deep-boltzmann machines, and a fast inference algorithm for spike-and-slab sparse coding while doing his PhD - he also led the development of Pylearn2 (the machine learning library for ML researchers), and contributed greatly to Theano. Cutting his teeth at Google then becoming Senior Research Scientist on the Google Brain team, Ian has now found his way to OpenAI - the non-profit research institution funded in part by Elon Musk and Peter Thiel and is working hard on developing breakthrough Deep learning techniques. Oh, he’s also lead author of a recently launched three part series available online co-written with Yoshua Bengio and Aaron Courville. We catch up with him.

WTB: Your book www.deeplearningbook.org covers Applied Math and Machine Learning Basics, explores Modern Practical Deep Networks, and Deep Learning Research. Who will benefit from this incredible knowledge source?

In the short term, I expect that software engineers who want to get involved in deep learning will benefit the most from the textbook.

In the long run, hopefully everyone will benefit, because more engineers will use deep learning and build smarter apps that help everyone, even people who don’t know that their app uses deep learning.

WTB: Personally, what’s most exciting about machine learning today?

I’m excited that machine learning now works well enough that we can focus on making it private and secure. There is a lot of work on resisting adversarial examples and on differential privacy.

WTB: With the information overload — how can we ensure efficient organisation and collaboration?

Within an institution, I think it’s important to keep teams small and focused, and to offer a nice low-bandwidth way for people to communicate between teams. At OpenAI we use Slack, and I think it works very well.

Across the whole AI research community, it’s actually very difficult to stay caught up with everything that’s going on. A few years ago, I felt like I actually knew absolutely everything that was happening within the field of deep learning, but now I don’t think that is feasible anymore. Every day, 4–5 new papers come out on ArXiv. I don’t think I even know everything being done with GANs. It’s important to talk to other people a lot, and find out which papers your friends think are really important.

WTB: Tell us about OpenAI Gym please

I don’t personally work on OpenAI Gym, but I can tell you about it anyway.

Most advances in AI have been triggered by the availability of better datasets, not the invention of a new algorithm (source: edge).

For reinforcement learning, we don’t need just a dataset, we need entire environments. Gym provides a unified framework for reinforcement learning environments, and also provides several specific environments, in order to provide the data necessary to spark the next advance in reinforcement learning.

WTB: What’s the most exciting part of your job?

The most exciting moment is when something suddenly works after weeks of it not working.

WTB: What advice would you give to budding AI research scientists and developers?

Focus on learning the fundamentals: good linear algebra, probability, and software engineering skills. The state of the art in machine learning changes from one year or even month to the next, but the fundamentals stay the same for decades.

WTB: What did you enjoy about speaking at AI With The Best 1st Edition and are you excited about September’s talk?

As a researcher, I’m excited about the potential of technology to transform society, but most of research-related institutions don’t actually make much use of technology. Most conferences are still held only in person and require everyone who attends to buy expensive plane tickets and release a lot of carbon into the atmosphere.

I like that AI With the Best uses the internet to bring everyone together, so there are fewer barriers to attendance and a truly global audience.

Thanks so much Ian for chatting with us!

You can catch Ian’s talk “Practical Methodology for Deploying Machine Learning” from last year’s edition of AI With The Best.

Find out more about Generative Adversarial Networks with Ian 11.20am Saturday 24th at AI With The Best online conference September 24–25th 2016