TensorFlow Post — Deep Learning with TensorFlow
Understanding data and arriving at a function:-
Neural Network is all about learning to arrive at a function from the data. It tries to understand the data, so that, given some input, it arrives at the output, using the learned function. For example, the input could be some set of raw pixels of an image and the output could be a word, ‘cat’.
Deep Learning involves automatically generating many layers of abstraction, during the course of learning the function. There are neurons in each layer, and each neuron implements a simple mathematical function. But the composition of millions and billions of these functions within this deep learning model is what makes it powerful.
Each neuron in a layer, is connected to a bunch of neurons. They form the layers in the models.Higher Layers can form interesting combination of lower level of features that have been detected.The neurons in the machine learning models, learn to cooperate to accomplish a complicated task.
Based on input raw set of pixels, some of the neurons in different layers will get activated and in the final layer, it might make a guess that its a ‘cat’. But with training, we make little adjustments of the neurons in different layers, so that finally, the model learns, in basically, the mathematical functions are adjusted, in order to almost correctly predict the correct answer.
Functions are simple — they are like matrix multiplier or vector-dot product. The composition of mathematical functions, upto millions and billions of these, form a powerful deep learning model, which helps in recognizing some insights from the input- be it an image, an audio, or a video or simple text. So, these models (mathematical functions, in turn) have lots of floating point operations, they are computationally intensive.
Important Property of Neural Networks:-
The results of the models tend to get better if we train them with
- more data
- bigger models
- more computation
New insights, Better algorithms and Good techniques always help to get amazing results from neural networks. But, most importantly, to work with large datasets, scaling is what is important at the core.
Reusability of machine learning models:-
If a machine learning model is arrived at in a domain, it could be reused in another domain. For example, while solving some image processing problems, if a machine learning model is arrived at, it could be used in speech processing domain, with little bit of training and arrive at good results.
It is a system built by Google to express machine learning ideas, train them, deploy them in real world products. It is open source, has an apache license 2.0, released in Nov 2015. Caffe, Theano, Torch are other machine learning frameworks which have been around quite a while.
TensorFlow has created a standard way to express machine learning ideas and computations. It is built out of DistBelief, older machine learning system, used within Google, scalable and production-deployable system, less flexible than TensorFlow.
Core of TensorFlow & Device-Variety:-
At the core, TensorFlow is a graph execution engine. It is designed in such a way, that you can run these graphs on a variety of devices. It seamlessly allows you to move from one device to another, for example, you would have trained your model in a data center with lot of GPUs, but would actually run the same model by deploying it in an app in the smartphone. The core of tensorflow is written in C++.
There are different front-ends which are driving the computation — currently, it has C++ Frontend and a Python Frontend. There are various front-ends that are in the works, majorly contributed by the open source community.
Basically, you build a graph to compute a neural network inference. mnist is kind of the ‘hello world’ of machine learning, the dataset where in there are bunch of images with letters and numbers and you build a machine learning model which recognize these letters and alphabets.
TensorFlow has a feature — which is automatic differentiation, for example, we would have a machine learning model and we would want to optimize it with respect to a loss function, basically, we want to minimize the loss function. So, you can symbolically, differentiate the graph, so that you will know what updates need to be done to your machine learning model, so that, next time, likely, the model will actually be able to predict that it is a ‘dog’. For example, you train the model with ‘gradient descent optimizer’ in order to minimize the ‘entropy loss’.
TensorFlow computation is nothing but a graph and what TensorFlow does, is that it executes this graph. So, basically, it’s a graph execution engine. Each node in the graph is an operation, as you see the circular node. And each edge which flow across these nodes are nothing but the tensors — multi-dimensional arrays with various kinds of types — like 2d array of floating point numbers, 3d array of integers.
TensorFlow is Distributed:-
The underlying system can decide that, this part of the computational graph, can be submitted to a GPU and another part of the graph can be computed by another different CPU, as shown in above image.
Decomposition of Graphs:-
TensorFlow seamlessly, inserts the send and receive nodes as part of the computation dataflow graph, so that some tensors are sent to different devices and some are received at different devices, these send/receive node/operation makes tensorflow distributed. And these send/receive operation manages all the communication underneath and we don’t get to really see them, as a user of the system.
Because of the ability to decompose the tensorflow computational dataflow graph, there is the possibility to run the same model on different kinds of machines, for example, smartphones, single machines (CPUs or GPUs), or a distributed systems of 100s of machines or GPU cards. OR you could have a custom machine learning hardware which is capable enough to perform the level of computation you need.
Some of the uses of Deep Learning:-
Other helpful literature related to TensorFlow:-
A MINDMAP RELATED TO UNDERSTANDING DATA:-
Originally published at www.iotenthu.com on August 29, 2016.