The idea of curriculum learning has already been proposed by Elman in 1993 and shown to improve network performance in several tasks by Bengio in 2009. Still, you hear surprisingly little about it these days which is why I decided to write a post about this.

Curriculum learning describes a type of learning in which you first start out with only easy examples of a task and then gradually increase the task difficulty. We humans have been learning according to this principle for decades, yet we don’t transfer it to neural networks and instead let them train on the whole data set with all its difficulties from the beginning on.

To demonstrate the effects of curriculum learning I will use a rather small convolutional neural network which tries to classify images into 10 categories. The data set is called CIFAR-10 and can be found here: https://www.cs.toronto.edu/~kriz/cifar.html


In many real world applications new training data becomes available after a network has already been trained. Especially with big neural networks, it would be very tedious to retrain the complete model every time new information becomes available. It would be much easier to simply add new nodes to the network for each new class or other information that was introduced and keep all the other previously trained weights. In this post I will give a brief overview on how to do this in TensorFlow and how the network performance is affected by it.

I will demonstrate the procedure and…


Animation of Heroin Deaths in the USA. Created in Python using Seaborn.

Matplotlib and Seaborn are some nice libraries in Python to create great looking plots. But these plots are all static and it’s hard to depict the change of data values in a dynamic and pleasingly looking way. How nice would it be if in you next presentation, video or social media post you can present development in the data by using a short video clip? And even better, you can still keep using Matplotlib, Seaborn or any other library that you like to use for your plots!

I’ve recently made a couple of dynamic plots for a short documentary about…

Viviane

Pursuing PhD in Cognitive Computing, 9+ years coding experience, Data Science & Machine Learning enthusiast, hobby film producer. vclay.ml

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store