Image for post
Image for post

Quick Overview of Theano: A Popular New Python Library for Deep Learning

In a recent engagement, we were tasked with evaluating sets of time series data at many intervals to determine the health of the system (essentially, looking to classify each increment of the time series). Being biased towards Python, we built datasets and began testing with most of the standard machine learning classification algorithms in sci-kit learn. It quickly became evident that we were not going to be able to achieve the desired level of predictive accuracy; it seemed that using lagged variables and other standard tricks did not truly capture the essence of the correlation between time periods.

After some discussion, we determined that the data could be modeled well using recurrent neural networks. Our first stop was at pybrain: one of the standard modules for building neural networks in Python. The library offered all of the flexibility required to build our custom network, but training on such a large network (containing over 100 hidden layers) took far too long.

Ultimately, we turned to a library that is making a lot of noise in the deep learning community: Theano. Theano has a steep learning curve for most Python users as the framework for declaring variables and building functions differ greatly from base Python. No machine learning algorithms exists “out-of-the-box” in Theano, however tutorials and examples exist to get new users started in the right direction. If you can get past these hurdles, the performance benefits are incredible. Many great Python libraries (numpy, pandas, etc.) perform operations more quickly than analogous operations in base Python by taking advantage of optimized methods ran in C or FORTRAN. Theano can dynamically generate C code for all functions generated within its framework allowing it “to attain speeds rivaling hand-crafted C implementations for problems involving large amounts of data”. In addition, Theano code can be executed on GPUs (which a typical PC may have hundreds of versus a small handful of CPUs), increasing speed by an order of magnitude by taking advantage of additional parallel processing. As a disclaimer, this was not a trivial task*.

It took us about a week to get our custom neural networks up and running on GPUs in Theano. In the end, the extra time invested proved worthwhile, as the several hundred models that we needed to build for cross-validation and parameter tuning over many time horizons took a little over two days to train; by our estimation, the same models built and executed in pybrain would have taken over a month. If you are considering building a complex neural network, or just training a standard net on a large set of data, take a look at the Theano tutorial; there is a good chance you will find that the initial time investment will pay off.

Note: if you do not have access to a machine with a GPU that is compatible with Theano, AWS instances can be used (you can even find some AMI’s with the necessary software for Theano already installed).

If you liked this blog post, check out more of our work, follow us on social media or join us for our free monthly Academy webinars.

The Opex Analytics Blog

Solving Complex Business Problems with Human and Artificial…

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store