Tensorflow Concepts for beginners with examples
Deep Learning is everywhere. Most of the beginners (and even expertsđ ) have been using Keras for creating Neural Networks and chilling their lives out!!
but with ease comes restriction.
Keras at times is rigid and it is the time when most of the Data Scientists move to the all-time great !!
TensorFlow
So, let's dive
1. Tensor: A n-dimensional array.
For example, a 1Ă1 tensor is a scalar, a 1Ăn tensor is a vector, an nĂn tensor is a matrix, and an nĂnĂn tensor is just a three-dimensional array.
2. Nodes: A node represents an operation, possibly applied to some input, and can generate an output that is passed on to other nodes.
Example: a=matmul(x,w).Here matmul() is a node
Nodes arenât calculated on the go but using Session object(discussed below)
3. Edges/Tensor objects: As we know nodes arenât calculated on the go. But the graph has to be created before calling the Session object. Hence the reference to their to-be-computed result can be passed onâflowâto another node while creating the graph. These references, which we call the edges/Tensor objects.
4. Computational Graph: A graph refers to a set of interconnected entities, commonly called nodes(discussed above) or vertices. The output of one node may flow to another node or the graph could stop as well. Even data generating functions are considered a node!!
The below image explains it better
5. Default_graph: If not mentioned separately, all variables, constants and other stuff get added to a single graph called the default_graph.
To create a different graph, create a new Graph() object and set it as default_graph.
6. Constants: They are like C/Java constant variables whose values canât be changed.
Example: a=tf.constant(1.0,dtype=tf.float32).You canât change the value of âaâ now!!
7. Variable: As the name suggests, they are the ones whose value can be changed.
Example: a=tf.variable(1,dtype=tf.float32).You can change âaâ.
8. Matmul & Multiply: matmul represents matrix multiplication while multiply is element-wise multiplication.
LHS:multiply()
RHS:matmul()
9. Reduce_mean: This is the same as np.mean used to calculate the mean of a tensor. The âaxisâ parameter can be used to calculate row-wise or column-wise mean of the tensor.
10. Session: A Session object is like an interface between Python objects and data on our end, and the actual computational system where memory is allocated for the objects we define, intermediate variables are stored, and finally results are fetched for us. In short, to execute a neural network, we need to create a session.
Have a look to understand Session object
11. Global_variable_initializer: In TensorFlow, all the variables & constants must be initialized before creating a session.Global_variable_initializer helps us to initialize all variables/constants associated with the current Graph at once.
12. Fetches: It refers to the variable/list_of_variables we want to calculate passed to session.run(). We must remember that no variable will be calculated if not mentioned in âfetchesâ or hasnât been used to calculate any of the âfetchesâ variable.
13. Logits: This can be seriously confusing as it has many meanings on the internet. In a Neural Network using Tensorflow, it refers to the Final Dense layer outputs before applying any activation function(mostly softmax)
14. Placeholders: They can be taken as containers for feeding various dynamic values to the neural network.
Example: We would want to train our model with batches of data. Since each batch is different, we canât stagnate the input. Hence with placeholders, we can achieve this. We will be passing each batch to the model.
Now Some Tough Onesđ˘
15. tf.nn._: Here the â_â can be replaced by any layer(dense,conv2d,etc.).It is a low level operation for neural network layers(conv2d,drop,etc.) , activation-functions like sigmoid,etc.It provides some flexibility over tf.Layers
16. tf.layers._: It can be taken as a Wrapper for tf.nn.Like in tf.nn, â_â can take various values. It internally calls tf.nn._
tf.layers automatically handles a lot of parameters like bias and weights while it all has to mentioned externally in tf.nn(as it is low level).But as mentioned tf.Layers can be rigid for certain parameters and hence reduces flexibility
17. tf.contrib: It is the module/codes contributed(with additional features) and would be soon in the core Tensorflow Library. Codes here might get changed in the future and there arenât any guarantees.
TRY AT YOUR OWN RISK
18. softmax_cross_entropy_with_logits: You will find this term quite often. Let's break this into 3 parts.
- Logits- Output of Dense Layer
- SoftMax:
here, theta represents logits.
- Cross_Entropy: Look below
Hence, softmax_cross_entropy_with_logits = Cross_entropy(softmax(logits))
19. feed_dict: When we are using placeholders in a tf model while using session.run(), we need to use feed_dict to pass values to those placeholders. Example:
20. Random_normal(shape, mean, std): This functionality returns a tensor of given shape from a normal distribution with a given mean and standard deviation.
21. Truncated_normal(shape, mean, std): This functionality returns a tensor of given shape from a truncated normal distribution with given mean and standard_deviation.From truncated, it is meant that extreme values are not included. The below picture will clear certain things.
The image on your left hand is Normal Distribution whereas the right-hand one is Truncated Normal Distribution
22. Variable_scope: Variable scope in Tensorflow is amongst the most useful concept which helps us to share tf variables. It creates a namespace(a pool) for all variables, constants, op(like +, -, *, etc) and helps us to avoid similar names in the graph and to reuse the predefined variables.
23. Name_scope: Quite similar to variable_scope, it is also helpful in serving the same purpose as variable_scope but with a difference. It doesnât account for variables created using tf.get_variable().
They both add a prefix to the variable name as shown below:
See the names of different variables used under different scopes.
24. tf.summary: This helps us to create log files regarding the training models in TensorFlow.For logging purposes, we need to create a FileWriter as well which intakes the log directory & computation graph.
To be specific, Summary is a special TensorBoard operation that takes in a regular tenor and outputs the summarized data to your disk (i.e. in the event file). More info can be found out here.
25. Tensorboard: Tensorboard in TensorFlow provides us with visualizing powers. You can use TensorBoard to visualize your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it. It simply uses the log files we have created using tf.summary.
This would be enough to keep you busy for the week!!!