Teaching To Learn: A Deep Learning Case Study
Wherein I selfishly teach a subject to learn it better and find my blind spots.
I’ve been taking a recent online deep learning course, Practical Deep Learning For Coders 1, due to fast.ai, a company that wants to make neural nets uncool again. Here’s the first part of my story in n* parts.
The instructors, Jeremy Howard and Rachel Thomas, have a unique teaching philosophy that I deeply appreciate as a programmer. You only understand something well if you can explain it to someone else in such a way that they can build what you explained. Better yet, if you can explain it to a computer by actually writing the code. So, three weeks after I started taking this course, I decided to apply the “teach to learn” philosophy.
On Saturday, Sept. 23rd, I gave a presentation at the Nairobi Women in Machine Learning and Data Science meetup, which was hosted at the Moringa School. Since the purpose of the talk was to practice what I’d just learned, I decided to demonstrate an interesting technique for prototyping a deep neural network, using an Excel spreadsheet. This is in lesson 2 of the first fast.ai course, in which we’re encouraged (enthusiastically) to prototype things quickly using Excel spreadsheets, to get a better understanding of the algorithmic details, without the added interface of numpy, Keras and Tensorflow/Theano code that are typically used in neural net explainers.
Not that those are problematic, it’s just that it’s very helpful to your intuition, to see explanations of ideas that seem abstract, and are very numerical, in many different kinds of media. While not exactly an explorable explanation, the spreadsheet I used to describe the details of a simple four-layer neural net is an excellent example of such a medium.
I also took the opportunity to show some code, while explaining how I’m participating in the challenges offered in the class. One of these, and what I showed, was part of my entry to the Dogs vs. Cats Redux challenge on Kaggle. The code for all of this is here, courtesy of fast.ai. This is a really cool example of the power of convolutional networks, which uses the former world-beating VGG16 neural net model to classify 2,000 images as either cats or dogs. Huddled together around a laptop, I walked people through the different code snippets that comprise a model that fine-tunes VGG16 to achieve 95% accuracy in the classification challenge.
All in all, I think that the talk fulfilled its stated purpose, for me, which was to explain something while keeping a look out for blind spots in my understanding. Here’s what I noticed from the experience.
One thing I’d skimmed when studying was the importance of the softmax function, which forces a neural network layer’s output values to range between 0 and 1, thus becoming a probability distribution.
Another was the fact that the probability you get from the neural net after this operation is applied isn’t a true probability distribution in the statistical sense, but a special kind that’s only applicable to that model’s training data.
Next lesson: diagrams are important, even when playing with different media for explanation. This really helped when demonstrating the IPython notebook from the course, which had lots of neat visualisation techniques typically used in data science, including confusion matrices and animated graphs. These can be found in the notebook here.
While no new results were shown except what people can run by simply cloning a github repository, I think that all 5 people who attended the meetup enjoyed the experience of seeing a well-known, surprisingly accurate visual recognition technique in action, and seeing the different hilarious ways it can fail to recognize a few errant examples.
Thanks to Kathleen Siminyu and Muthoni Wanyoike for reading a draft of this.
