Interview with OpenAI Fellow: Christine McLeavey Payne

Part 8 of The series where I interview my heroes.

Index and about the series“Interviews with ML Heroes”

Today, I’m honored to be talking with Christine McLeavey.

Christine is currently working at OpenAI as a member of the Technical Staff (Fellow), She’s also a Physicist and holds a Master’s degree from Stanford in Neuroscience and Medicine.

She’s also an artist: a Professional Pianist.

About the Series:

I have very recently started making some progress with my Self-Taught Machine Learning Journey. But to be honest, it wouldn’t be possible at all without the amazing community online and the great people that have helped me.

In this Series of Blog Posts, I talk with People that have really inspired me and whom I look up to as my role-models.

The motivation behind doing this is, you might see some patterns and hopefully you’d be able to learn from the amazing people that I have had the chance of learning from.

Sanyam Bhutani:​ Hello Christine, Thank you so much for doing this interview.

Christine McLeavey: Thanks Sanyam. I’m honored you asked me to be here.

Sanyam Bhutani:​ You’ve had a very interesting career path. You’re a Physicist, a Neuroscientist and also an artist!

Can you tell us how Deep Learning came into the picture, What got you interested in Deep Learning?

Christine McLeavey: I actually discovered Deep Learning a bit by chance. As a pianist, I was always in need of someone to turn my sheet music pages for me. I decided to keep all my music on my laptop and I wrote a program that used an eye tracker to know when to turn my pages automatically. As I looked into ways to shift this system over to an iPad, I kept hearing about something called “TensorFlow”. Of course, at the time I knew nothing at all about machine learning. I took Andrew Ng’s Coursera courses on machine learning and deep learning, and loved those so much that I continued with the FastAI sequence. I realized Deep Learning ties together so many of my interests (math, neuroscience, medicine, and even a lot of creativity). It’s the perfect fit for me.

Sanyam Bhutani: Today you’re working at OpenAI as a fellow.

How would you describe a day at OpenAI? What do your daily activities look like?

Christine McLeavey: The fellows program starts out with a two month “curriculum”, where you work through papers and projects teaching you the cutting edge of RL, Generative Methods, Attention, and other key topics. I’m right in the midst of this, so most of what I’m doing is fairly independent. After this, I’ll start into a project with the multiagent group, so at that point I’ll be collaborating with others.

Sanyam Bhutani: Could you tell us more about your Project at OpenAI: Clara?

Christine McLeavey: I created Clara as my OpenAI scholars program final project this past summer. I trained an LSTM on classical and jazz piano music, and then used that to generate new samples. I was quite excited about the early results — I have some sample clips here: http://christinemcleavey.com/human-or-ai/ . I’m now joining up with the language and safety teams here at OpenAI to see if we can push these results further (trying out different language models, and incorporating human preferences).

Sanyam Bhutani: It’s quite interesting to see that as an artist you’re trying to create Neural nets that replicate human creativity.

From an Artist’s perspective, What are your thoughts about making a creative “AI” ?

Christine McLeavey: That’s an interesting question. On one hand, it’s hard to see an LSTM as creative. Behind the scenes, my model is just predicting how likely it thinks each note is for the upcoming step. It then randomly chooses between those notes, based on that probability. I can increase the randomness and maybe it will seem more creative, but it’s still just based on patterns in the training data.

On the other hand, maybe human creativity is similar in ways. We all have building blocks that we put together into patterns — often usual, predictable ones, and then sometimes wacky random ones. Maybe the most important thing about human creativity is our ability to recognize the times when our wacky and random experiments are actually beautiful art.

I’m excited to see how “creative AI” will start interacting with human emotions. Can a music generator make a piece that makes me feel sad? Can a neural net decode the news, and create a painting that captures our reaction to current events?

Sanyam Bhutani: What development in Deep Learning are you most excited about?

Christine McLeavey: There’s so much going on right now that it’s hard to pick one thing. Personally I’m excited about developments in natural language and in generative models. On the Reinforcement Learning side, I find multi-agent work to be very interesting — studying how competition and cooperation between different agents pushes each one to learn new skills. Lastly, I’m excited that it’s getting easier for people from other fields to learn about deep learning. I think there will be many developments that can directly benefit society as doctors, lawyers, environmentalists, and so many others can use deep learning tools in their own fields.

Sanyam Bhutani: For the readers and the beginners who are interested in working on Deep Learning and dream of joining the OpenAI Scholars program. What would be your best advice?

Christine McLeavey: I think now is such an exciting time to begin this journey. There are so many opportunities (at OpenAI, but also more broadly), and everything you need to learn is available online for free or next to free.

In terms of specifics — one of my favorite pieces of advice was from FastAI’s Jeremy Howard. He told me to focus on making one really great project that showed off my skills. He says that so many people have many interesting but half-finished projects on github, and that it’s much better to have a single one that is really polished.

The other thing I’d add is that I’ve learned the most about deep learning when I’ve taught it to others. I became a mentor for Andrew Ng’s Coursera courses, and trying to answer other students’ questions pushed me to understand the material at a much deeper level. Whenever you learn a new topic, try explaining it to someone else (this’ll also be great preparation for job interviews!).

Sanyam Bhutani: How do you stay up to date with the cutting edge?

Christine McLeavey: Being at OpenAI certainly helps a lot, since we frequently have meetings where one person will present a few recent papers and then we’ll discuss those. I’m also now a big fan of twitter, and I’ll often see a paper or idea mentioned there.

To push myself, I keep a notebook where each day I write down a quick summary of one new topic I learned that day. Some days it’s a big topic and some days a little detail, but it has to be something every single day.

Sanyam Bhutani: Do you feel Machine Learning has been overhyped?

Christine McLeavey: What worries me more are the times when people put too much trust in machine learning results (particularly when it comes to things like filtering through job applications, or guessing whether a past criminal will commit a future crime). We’re starting to pay more attention to the ways models can be biased and otherwise flawed, but I don’t think that awareness and caution has reached the general public yet. I think Machine Learning is an extremely exciting tool, and that there are amazing possibilities we haven’t dreamed of yet, but that we need to remember it’s only a tool.

Sanyam Bhutani: Being from a “Non-Computer Science background”

What are your thoughts about the generally prevalent pre-req of having a Ph.D. or Masters level of expertise to able to contribute to the field of Deep Learning?

Christine McLeavey: Well, technically I’ve only taken one computer science course for credit in my life (Intro to CS back in college), so I’m strongly biased to say a PhD shouldn’t be a pre-req! I did have a strong math background, and that definitely helped me feel more confident as I started in deep learning.

A main thing I’m learning now at OpenAI (and which I missed by not doing a PhD) is a sense of scientific discipline. I love to experiment and to imagine new ideas, but I’m learning now to create more rigorous experiments, and to spend more time reading papers.

I don’t think a Masters or PhD is at all necessary to understanding or contributing to the field of Deep Learning. However, I’m continually awed by the depth of knowledge of people who have been in the field for a while. My best advice is to find a good mentor who has this long experience, and can help you see the current trends and to know what you should study and where you can fit in.

Sanyam Bhutani: Before we conclude, any advice for the beginners who even though are excited about the field, feel overwhelmed to even get started with Deep Learning?

Christine McLeavey: I would say: embrace that feeling of being overwhelmed. So often, when I first tackle a new subject, I have the feeling that all the ideas are jumbled in my head and that I’m not really understanding anything. Nowadays, I’ve gone through this so many times, that I recognize that feeling of being overwhelmed, and I’ve learned not to give up at that point. I know that when I come back to the material in a day or a week, it will make more sense.

In terms of where to begin, I’m a huge fan of online courses. I wrote up a page here http://christinemcleavey.com/learning-about-deep-learning/ that has my favorite online resources.

My main advice is just to dive in. Things probably won’t make sense the first time around, but that’s ok. Don’t assume the problem is you! Sometimes an author or a teacher isn’t being clear, so look around to see if a blog post or a different video explains things in a different way. Also, don’t only read through or copy example code — try changing it and experimenting. As you circle back to ideas again, the pieces will start to fit together. Good luck!

Sanyam Bhutani: Thank you so much for doing this interview.

If you found this interesting and would like to be a part of My Learning Path, you can find me on Twitter here.

If you’re interested in reading about Deep Learning and Computer Vision news, you can checkout my newsletter here.