Ruby Tensorflow for developers
In the previous part of this tutorial, I introduced a bit of Ruby TensorFlow and showed a few examples of using Ruby Tensorflow.
In this part, I will dwell upon simple and useful ideas that will help developers understand how google protobuf is used in Ruby Tensorflow. Even though the primary purpose of the blog post is to explain the Ruby API, I am sure that developers specializing in different languages can greatly benefit from the ideas here.
Google Protobuf
Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. The method involves an interface description language that describes the structure of some data and a program that generates source code from that description for generating or parsing a stream of bytes that represents the structured data.
It helps define data structures in text files, and the protobuf tools generate classes in Ruby, C, Python, and other languages that can load, save, and access the data in a friendly way. Google developed Protocol Buffers for use internally and has provided a code generator for multiple languages. In this blog, I will focus on ruby protobuf so to start it’s worth getting familiar with how they work.
GraphDef
The foundation of computation in TensorFlow is the Graph object. This holds a network of nodes, each representing one operation, connected to each other as inputs and outputs.
The GraphDef class is an object created by the ProtoBuf library from the definition in tensorflow/core/framework/graph.proto. The protobuf tools parse this text file, and generate the code to load, store, and manipulate graph definitions. If you see a standalone TensorFlow file representing a model, it’s likely to contain a serialized version of one of these GraphDef objects saved out by the protobuf code.
This generated code is used to save and load the GraphDef files from disk. A good example to look at as we dig into this is graph_metrics.py. This Python script takes a saved graph definition, and analyzes the model to estimate performance and resource statistics. The code that actually loads the model looks like this:
require 'tensorflow'
graph_def = Tensorflow::GraphDef.newThis line creates an empty GraphDef object, the class that’s been created from the textual definition in graph.proto. This is the object we’re going to populate with the data from our file in irb.
reader = File.read('graph.pb')
graph_def = Tensorflow::GraphDef.parse(reader)
graph_def.node[0].name
=> "input1"
graph_def.node[1].name
=> "input2"
graph_def.node[2]
=> <Tensorflow::NodeDef: name: "output", op: "Add", input: ["input1", "input2"], device: "", attr: {"T"=><Tensorflow::AttrValue: list: nil, s: "", i: 0, f: 0.0, b: false, type: :DT_INT64, shape: nil, tensor: nil, placeholder: "", func: nil>}>Nodes
Once you’ve loaded a file into the graph_def variable, you can now access the data inside it. For most practical purposes, the important section is the list of nodes stored in the node member.
Each node is a NodeDef object, also defined in graph.proto. These are the fundamental building blocks of TensorFlow graphs, with each one defining a single operation along with its input connections. Here are the members of a NodeDef, and what they mean.
Name: Every node should have a unique identifier that’s not used by any other nodes in the graph. The name is used when defining the connections between nodes, and when setting inputs and outputs for the whole graph when it’s run.
op: This defines what operation to run, for example “Add”, “MatMul”, or “Conv2D”. When a graph is run, this op name is looked up in a registry to find an implementation. To have a better understanding of Tensorflow ops take a look at ops.pbtxt.
input: A list of strings, each one of which is the name of another node, optionally followed by a colon and an output port number.
attr: This is a key/value store holding all the attributes of a node. These are the permanent properties of nodes, things that don’t change at runtime such as the size of filters for convolutions, or the values of constant ops. Because there can be so many different types of attribute values, from strings, to ints, to arrays of tensor values, there’s a separate protobuf file defining the data structure that holds them, in tensorflow/core/framework/attr_value.proto.
Each attribute has a unique name string, and the expected attributes are listed when the operation is defined. If an attribute isn’t present in a node, but it has a default listed in the operation definition, that default is used when the graph is created.
Text or Binary?
There are actually two different formats that a ProtoBuf can be saved in. Text Format is a human-readable form, which makes it nice for debugging and editing, but can get large when there’s numerical data like weights stored in it. You can see a small example of that in graph_run_run2.pbtxt.
Binary format files are a lot smaller than their text equivalents, even though they’re not as readable for us. In this script, we ask the user to supply a flag indicating whether the input file is binary or text, so we know the right function to call. You can find an example of a large binary file inside the inception_dec_2015.zip archive , as tensorflow_inception_graph.pb (This file is used on image recognition tutorial). In ruby protobuf only binary format is supported which makes it a little more difficult for developers to use it but I have added a simple way to achieve back and forth conversion from binary to human readable format using take a look at this file.
Example
Now let's took a look at a simple example to understand to make use of all the ideas I have mentioned above. But before proceeding further, you must have a completely working tensorflow.rb and tensorflow installed.
Now let's start with python
This creates a simple .proto file
All the necessary files to make use of Tensorflow with google Protobuf are here. Now, I will try to make the same file in Ruby but we will do that manually
You can see how I have manually defined every node and attribute here. The only problem with this file is that this creates the protobuf in binary wire format so you cannot read it. Therefore, we need to transform the file into human readable format for which you can use this file
Now you can look at the same file built by ruby in human readable format.
I understand that this example may be trivial at first glance, but it does highlight how you can easily use Google protobuf.The list of operations perfectly defined here: ops.pbtxt so you can read it and get a better understanding. I encourage everyone to play with the ruby protobuf and also take a second look. To make it easier to work with ops I made a generic function that helps to define ops but its obvious that things can be improved.
To give you a better understanding, I am going to suggest a simple way on how you can make many graphs in ruby and understand them? Go to the specs (for example math spec) and just after session.extend_graph(graph) the add another line
File.open(‘graph.pb’,’w’) {|file| file.write(graph.graph_def_raw)} And run the specs again. This will save the graph definition in a graph.pb file and then you can convert it using pb_to_pbtxt file(python).
With this, I will mark the completion of this blog. I hope you find this interesting and useful. If you find any thing conflicting feel free to comment below.
Acknowledgements
Special Thanks to Jason toy (Founder somatic), Soon Hin Khor (Phd. Univ. of Tokyo) and Sameer Deshmukh (Founder Daru and member of Sciruby) for being the the mentors for this project. They have been very supportive to me with everything, and I am very grateful to them.
