Theano : a computational library

Written by : Febrian Hendifa, Muhammad Julizar, Venu Fitratama & Wenny Chandra Sari

What is Theano?

Theano is an open source library project that provides Python for performing mathematical operations on multidimensional arrays. Using Theano we can create and develop deep learning models or wrapper libraries. Theano computations are expressed using a NumPy-esque syntax on either using a GPU (Graphic Processing Unit) or CPU (Central Processing Unit). which can increase productivity to a higher level. Theano is developed by the Montreal Institute for Learning Algorithms (MILA) at the Université de Montréal.

The actual syntax of Theano expressions is symbolic, which can be off putting to beginners used to normal software development. Specifically, expressions are defined in the abstract sense, compiled and later actually used to make calculations.

It was specifically designed to handle the types of computation required for large neural network algorithms used in Deep Learning. It was one of the first libraries of its kind (development started in 2007) and is considered an industry standard for Deep Learning research and development.

Why Should We Use Theano?

For the purpose of modifying and evaluating expressions, particularly matrix-valued ones, Theano is a Python library and optimizing compiler. What does Theano achieve that Python and numpy do not, when manipulating matrices is generally done with the numpy package?

• execution speed optimizations: Theano can use g++ or nvcc to compile parts of your expression graph into CPU or GPU instructions, which run much faster than pure Python.
• symbolic differentiation: Theano can automatically build symbolic graphs for computing gradients.
• stability optimizations: Theano can recognize [some] numerically unstable expressions and compute them with more stable algorithms.
• Transparent use of a GPU: perform data-intensive computations up to 140x faster than on a CPU (support for float32 only).
• Extensive unit-testing and self-verification: includes tools for detecting and diagnosing bugs and/or potential problems.
• The closest Python package to Theano is sympy but.. Theano focuses more on tensor expressions than Sympy, and has more machinery for compilation. Sympy has more sophisticated algebra rules and can handle a wider variety of mathematical operations (such as series, limits, and integrals).
• If numpy is to be compared to MATLAB and sympy to Mathematica, Theano is a sort of hybrid of the two which tries to combine the best of both worlds.

Overall, here are the advantages of using Theano:

• Theano built packages like Keras, Lasagne, and Blocks.
• Raw Theano is a low-level product.
• Having high-level modules like Keras, Blocks, Lasagne, etc, makes it more usable.

While these are some drawbacks of using Theano:

• On AWS, it can be complex.
• It can be run on a single GPU.
• Requires large compile time for vast and complex models.
• Error notices are complex, which makes debugging harder.

Theano Vs Tensorflow

• Theano performs tasks way faster than TensorFlow. Mainly the tasks that require a single GPU will run faster in Theano.
• Theano has native windows support while TensorFlow lacks of native windows. Theano also supports High-Level Wrappers like Lasagne.
• Theano has more documentation compared to TensorFlow.
• TensorFlow runs specifically on Linux, macOS, Windows, and Android while Theano runs on cross-platform.
• TensorFlow is one of the famous Deep Learning libraries and is mostly used for research purposes while Theano is an old Framework that is not used mostly.
• Theano is compatible with a deep learning library called Keras which contains some pre-trained models.

Theano Installation

Before you install Theano, you must install others library. anyway theano can be installed on Windows, macOs and Linux, The following is the list of others library

• Python
• NumPy − Required
• SciPy − Required only for Sparse Matrix and special functions
• BLAS − Provides standard building blocks for performing basic vector and matrix operations

There is another optional packages that you can choose to install depending on your needs:

• nose : To run Theano’s test-suite
• Sphinx − For building documentation
• Graphiz and pydot − To handle graphics and images
• NVIDIA CUDA drivers − Required for GPU code generation/execution
• libgpuarray − Required for GPU/CPU code generation on CUDA and OpenCL devices

To install Theano as a user, we can use

$ pip install Theano

After Theano is installed, we need to import Theano library to use in our program. To import it, we can use

from theano import *

Demo

User can use these simple command to see the use of Theano itself

!pip install Theano

import theano

from theano import tensor

# Declaring Variables

a = tensor.dscalar()

b = tensor.dscalar()

# Defining Expression

c = a + b

# Defining Theano Function

f = theano.function([a,b], c)

#Invoking Theano Function

d = f(3.5, 5.5)

#Output

print (d)

The output will be

9.0

Conclusion

The Machine Learning model building involves intensive and repetitive computations involving tensors. These require intensive computing resources. As a regular compiler would provide the optimizations at the local level, it does not generally produce a fast execution code.

Theano first builds a computational graph for the entire computation. As the whole picture of computation is available as a single image during compilation, several optimization techniques can be applied during pre-compilation and that’s exactly what.