TensorFlow: Tensor, Operation and Variable
In this example we will see how these elements are connected
This article shows differences and similarities between the following elements. It shows some output code for different operations.
(See and clone this article in a Jupyter Notebook if you want to play with it.)
- Tensor: ‘Represents one of the outputs of an Operation.’.
- Operation: ‘Represents a graph node that performs computation on tensors.’
- Variable: ‘A variable maintains state in the graph across calls to run().’
To get a variable we will use tf.get_variable ('Gets an existing variable with these parameters or create a new one.').
If wee need to reimplement the graph, run each time ops.reset_default_graph().
In TensorFlow:
- An Operation is some numerical operation happening to Tensors, it has outputs that are Tensors.
- A Tensor is kind of an operation’s output waiting to happen, that is until the op gets evaluated (
.eval()). - A Variable is kind of a special Tensor. ‘When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value’.
Note: To see the examples in this notebook at least run “Import stuff” and “Create the graph”, then the other examples can be executed without problem.
TOC (top)
- Import Stuff
- Create the Graph
- Output legend
varA and ‘a’, see the graph before.varB and ‘b’, see the graph before.timesAB and ‘a_mult_b’, see the graph before.addAB and ‘a_add_b’, see the graph before.
Import Stuff (top)
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow.python.framework import opsCreate the graph (top)
ops.reset_default_graph()varA = tf.get_variable(‘a’, initializer=2.) # ‘a’ is the name of the operation
varB = tf.get_variable(‘b’, initializer=3.) # ‘b’ is the name of the operationtimesAB = tf.multiply(varA, varB, name=’a_mult_b’) # ‘a_mult_b’ is the name of the multiply operation
addAB = tf.add(varA, varB)
tf.identity(addAB, name=’a_add_b’) # ‘a_add_b’ is the name of the identity operationinit = tf.global_variables_initializer()
Output legend (top)
In the following sections only the output of the code will be shown, it will use the following legend:
LEGEND:
----------------------------------------
group name
----------------------------------------
... something to be printed next
___ a comment
END LEGENDvarA and ‘a’ (top)
varA = tf.get_variable('a', initializer=2.) does this:
- Creates a Python variable called varA of type Variable. A kind of tensor.
- Creates an operation called ‘a’ which outputs a Tensor called ‘a:0’.
- Sets the initial value of varA or the Tensor ‘a:0’ to 2.0, this will only be initialized after
sess.run(init)is called.
In the code bellow (and its output) you can see how you can use varA, sess.graph.get_operation_by_name and sess.graph.get_tensor_by_name. The three are connected.
----------------------------------------
variable varA
----------------------------------------
... varA:
<tf.Variable 'a:0' shape=() dtype=float32_ref>
... varA.name:
a:0
... type(varA):
<class 'tensorflow.python.ops.variables.Variable'>
----------------------------------------
sess.graph.get_operation_by_name('a')
----------------------------------------
... sess.graph.get_operation_by_name('a'):
name: "a"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
... sess.graph.get_operation_by_name('a').name:
a
... type(sess.graph.get_operation_by_name('a')):
<class 'tensorflow.python.framework.ops.Operation'>
... sess.graph.get_operation_by_name('a').outputs:
[<tf.Tensor 'a:0' shape=() dtype=float32_ref>]
----------------------------------------
sess.graph.get_tensor_by_name('a:0')
----------------------------------------
___ sess.graph.get_tensor_by_name('a'):
ERROR, there isn't a tensor named 'a', use ':0'
... sess.graph.get_tensor_by_name('a:0'):
Tensor("a:0", shape=(), dtype=float32_ref)
... sess.graph.get_tensor_by_name('a:0').name:
a:0
... type(sess.graph.get_tensor_by_name('a:0')):
<class 'tensorflow.python.framework.ops.Tensor'>
----------------------------------------
object equivalence
----------------------------------------
___ the following are equivalent:
sess.graph.get_operation_by_name('a').outputs[0]
sess.graph.get_tensor_by_name('a:0')
___ but differ from:
varA
... sess.graph.get_operation_by_name('a').outputs[0]:
Tensor("a:0", shape=(), dtype=float32_ref)
... sess.graph.get_tensor_by_name('a:0'):
Tensor("a:0", shape=(), dtype=float32_ref)
... varA
<tf.Variable 'a:0' shape=() dtype=float32_ref>
----------------------------------------
object evaluation .eval()
----------------------------------------
___ eval(), the following are equivalent:
varA.eval():
sess.graph.get_tensor_by_name('a:0').eval()
sess.graph.get_operation_by_name('a').outputs[0].eval
... varA.eval():
2.0
... sess.graph.get_tensor_by_name('a:0').eval():
2.0
... sess.graph.get_operation_by_name('a').outputs[0].eval():
2.0
----------------------------------------
previous operation
----------------------------------------
... varA.op:
name: "a"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
... sess.graph.get_tensor_by_name('a:0').op
name: "a"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
... sess.graph.get_operation_by_name('a').outputs[0].op
name: "a"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}varB and ‘b’ (top)
varB = tf.get_variable('b', initializer=2.) does this:
- Creates a Python variable called varB of type Variable. A kind of tensor.
- Creates an operation called ‘b’ which outputs a Tensor called ‘b:0’.
- Sets the initial value of varB or the Tensor ‘b:0’ to 2.0, this will only be initialized after
sess.run(init)is called.
In the code bellow (and its output) you can see how you can use varB, sess.graph.get_operation_by_name and sess.graph.get_tensor_by_name. The three are connected.
----------------------------------------
variable varB
----------------------------------------
... varB:
<tf.Variable 'b:0' shape=() dtype=float32_ref>
... varB.name:
b:0
... type(varB):
<class 'tensorflow.python.ops.variables.Variable'>
----------------------------------------
sess.graph.get_operation_by_name('b')
----------------------------------------
... sess.graph.get_operation_by_name('b'):
name: "b"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
... sess.graph.get_operation_by_name('b').name:
b
... type(sess.graph.get_operation_by_name('b')):
<class 'tensorflow.python.framework.ops.Operation'>
... sess.graph.get_operation_by_name('b').outputs:
[<tf.Tensor 'b:0' shape=() dtype=float32_ref>]
----------------------------------------
sess.graph.get_tensor_by_name('b:0')
----------------------------------------
___ sess.graph.get_tensor_by_name('b'):
ERROR, there isn't a tensor named 'b', use ':0'
... sess.graph.get_tensor_by_name('b:0'):
Tensor("b:0", shape=(), dtype=float32_ref)
... sess.graph.get_tensor_by_name('b:0').name:
b:0
... type(sess.graph.get_tensor_by_name('b:0')):
<class 'tensorflow.python.framework.ops.Tensor'>
----------------------------------------
object equivalence
----------------------------------------
___ the following are equivalent:
sess.graph.get_operation_by_name('b').outputs[0]
sess.graph.get_tensor_by_name('b:0')
___ but differ from:
varB
... sess.graph.get_operation_by_name('b').outputs[0]:
Tensor("b:0", shape=(), dtype=float32_ref)
... sess.graph.get_tensor_by_name('b:0'):
Tensor("b:0", shape=(), dtype=float32_ref)
... varB
<tf.Variable 'b:0' shape=() dtype=float32_ref>
----------------------------------------
object evaluation .eval()
----------------------------------------
___ eval(), the following are equivalent:
varB.eval():
sess.graph.get_tensor_by_name('b:0').eval()
sess.graph.get_operation_by_name('b').outputs[0].eval
... varB.eval():
3.0
... sess.graph.get_tensor_by_name('b:0').eval():
3.0
... sess.graph.get_operation_by_name('b').outputs[0].eval():
3.0
----------------------------------------
previous operation
----------------------------------------
... varB.op:
name: "b"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
... sess.graph.get_tensor_by_name('b:0').op
name: "b"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}
... sess.graph.get_operation_by_name('b').outputs[0].op
name: "b"
op: "VariableV2"
attr {
key: "container"
value {
s: ""
}
}
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
}
}
}
attr {
key: "shared_name"
value {
s: ""
}
}timesAB and ‘a_mult_b’ (top)
timesAB = tf.multiply(varA, varB, name='a_mult_b') does this:
- Creates a Python variable called timesAB of type Tensor.
- Creates an operation called ‘a_mult_b’ which outputs a Tensor called ‘a_mult_b:0’.
- Sets this operation’s name to ‘a_mult_b’.
In the code bellow (and its output) you can see how you can use timesAB, sess.graph.get_operation_by_name and sess.graph.get_tensor_by_name. The three are connected.
----------------------------------------
variable timesAB
----------------------------------------
... timesAB:
Tensor("a_mult_b:0", shape=(), dtype=float32)
... timesAB.name:
a_mult_b:0
... type(timesAB):
<class 'tensorflow.python.framework.ops.Tensor'>
----------------------------------------
sess.graph.get_operation_by_name('a_mult_b')
----------------------------------------
... sess.graph.get_operation_by_name('a_mult_b'):
name: "a_mult_b"
op: "Mul"
input: "a/read"
input: "b/read"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
... sess.graph.get_operation_by_name('a_mult_b').name:
a_mult_b
... type(sess.graph.get_operation_by_name('a_mult_b')):
<class 'tensorflow.python.framework.ops.Operation'>
... sess.graph.get_operation_by_name('a_mult_b').outputs:
[<tf.Tensor 'a_mult_b:0' shape=() dtype=float32>]
----------------------------------------
sess.graph.get_tensor_by_name('a_add_b:0')
----------------------------------------
___ sess.graph.get_tensor_by_name('a_mult_b'):
ERROR, there isn't a tensor named 'a_mult_b', use ':0'
... sess.graph.get_tensor_by_name('a_mult_b:0'):
Tensor("a_mult_b:0", shape=(), dtype=float32)
... sess.graph.get_tensor_by_name('a_mult_b:0').name:
a_mult_b:0
... type(sess.graph.get_tensor_by_name('a_mult_b:0')):
<class 'tensorflow.python.framework.ops.Tensor'>
----------------------------------------
object equivalence
----------------------------------------
___ the following are equivalent:
timesAB
sess.graph.get_operation_by_name('a_mult_b').outputs[0]
sess.graph.get_tensor_by_name('a_mult_b:0')
... timesAB:
Tensor("a_mult_b:0", shape=(), dtype=float32)
... sess.graph.get_operation_by_name('a_mult_b').outputs[0]:
Tensor("a_mult_b:0", shape=(), dtype=float32)
... sess.graph.get_tensor_by_name('a_mult_b:0'):
Tensor("a_mult_b:0", shape=(), dtype=float32)
----------------------------------------
object evaluation .eval()
----------------------------------------
___ eval(), the following are equivalent:
timeAB.eval()
sess.graph.get_tensor_by_name('a_mult_b:0').eval()
sess.graph.get_operation_by_name('a_mult_b').outputs[0].eval()
... timeAB.eval():
6.0
... sess.graph.get_tensor_by_name('a_mult_b:0').eval():
6.0
... sess.graph.get_operation_by_name('a_mult_b').outputs[0].eval():
6.0
----------------------------------------
previous operation
----------------------------------------
... timesAB.op:
name: "a_mult_b"
op: "Mul"
input: "a/read"
input: "b/read"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
... sess.graph.get_tensor_by_name('a_mult_b:0').op
name: "a_mult_b"
op: "Mul"
input: "a/read"
input: "b/read"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
... sess.graph.get_operation_by_name('a_mult_b').outputs[0].op
name: "a_mult_b"
op: "Mul"
input: "a/read"
input: "b/read"
attr {
key: "T"
value {
type: DT_FLOAT
}
}addAB and ‘a_add_b’ (top)
addAB = tf.add(varA, varB)
tf.identity(addAB, name='a_add_b') # 'a_add_b' is the name of the identity operationdoes this:
- Creates a Python variable called addAB of type Tensor.
- Creates an operation called ‘a_add_b’ which outputs a Tensor called ‘a_add_b:0’.
- Sets this operation’s name to ‘a_add_b’.
- Note that since we are using tf.identity, the previous operation differs for addAB, sess.graph.get_operation_by_name(“a_add_b”). But is equal for sess.graph.get_operation_by_name(“a_add_b”) and sess.graph.get_tensor_by_name(‘a_add_b:0’).
In the code bellow (and its output) you can see how you can use addAB, sess.graph.get_operation_by_name and sess.graph.get_tensor_by_name. The three are connected.
----------------------------------------
variable addAB
----------------------------------------
... addAB:
Tensor("Add:0", shape=(), dtype=float32)
... addAB.name:
Add:0
... type(addAB):
<class 'tensorflow.python.framework.ops.Tensor'>
----------------------------------------
sess.graph.get_operation_by_name('a_add_b')
----------------------------------------
... sess.graph.get_operation_by_name('a_add_b'):
name: "a_add_b"
op: "Identity"
input: "Add"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
... sess.graph.get_operation_by_name('a_add_b').name:
a_add_b
... type(sess.graph.get_operation_by_name('a_add_b')):
<class 'tensorflow.python.framework.ops.Operation'>
... sess.graph.get_operation_by_name('a_add_b').outputs:
[<tf.Tensor 'a_add_b:0' shape=() dtype=float32>]
----------------------------------------
sess.graph.get_tensor_by_name('a_add_b:0')
----------------------------------------
___ sess.graph.get_tensor_by_name('a_add_b'):
ERROR, there isn't a tensor named 'a_add_b', use ':0'
... sess.graph.get_tensor_by_name('a_add_b:0'):
Tensor("a_add_b:0", shape=(), dtype=float32)
... sess.graph.get_tensor_by_name('a_add_b:0').name:
a_add_b:0
... type(sess.graph.get_tensor_by_name('a_add_b:0')):
<class 'tensorflow.python.framework.ops.Tensor'>
----------------------------------------
object equivalence
----------------------------------------
___ the following are (kind of) equivalent:
addAB
sess.graph.get_operation_by_name('a_add_b').outputs[0]
sess.graph.get_tensor_by_name('a_add_b:0')
... addAB:
Tensor("Add:0", shape=(), dtype=float32)
... sess.graph.get_operation_by_name('a_add_b').outputs[0]:
Tensor("a_add_b:0", shape=(), dtype=float32)
... sess.graph.get_tensor_by_name('a_add_b:0'):
Tensor("a_add_b:0", shape=(), dtype=float32)
----------------------------------------
object evaluation .eval()
----------------------------------------
___ eval(), the following are equivalent:
timeAB.eval()
sess.graph.get_tensor_by_name('a_add_b:0').eval()
sess.graph.get_operation_by_name('a_add_b').outputs[0].eval()
... timeAB.eval():
5.0
... sess.graph.get_tensor_by_name('a_add_b:0').eval():
5.0
... sess.graph.get_operation_by_name('a_add_b').outputs[0].eval():
5.0
----------------------------------------
previous operation
----------------------------------------
... addAB.op:
name: "Add"
op: "Add"
input: "a/read"
input: "b/read"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
... sess.graph.get_tensor_by_name('a_add_b:0').op
name: "a_add_b"
op: "Identity"
input: "Add"
attr {
key: "T"
value {
type: DT_FLOAT
}
}
... sess.graph.get_operation_by_name('a_add_b').outputs[0].op
name: "a_add_b"
op: "Identity"
input: "Add"
attr {
key: "T"
value {
type: DT_FLOAT
}
}