Introduction to TensorFlow for Developers. Part 1/?. Tensor multiplication

Machine Learning is where all the hype right now (and blockchain). We heare the worls like AlphaGo, Watson, Alexa recently Google Duplex thrown around on every conference. Trend that is unlikely to slow down. The vast amounts of data simply can’t be analized and used any other way. And certain tasks can not be performed by declaratively coding them in one of the programing language. I mean they can because varying nature of the outcomes forces us to use probability rather than determinizm. And at the risk of sounding too simplistic machine learning really is the a statistical analysis of the data. To that end we have a lot of tools matlab, R (separate language), varius libraries for all programing languages. In that whole eco-system two are most widely adopted: TensorFlow and PyTorch. As title sugest i will cover TensorFlow in these series.

In this series i promes you that we will have no or very little of machine learning. I know you might be disappointed but the truth is that the only reason I decided to write this series is because all the tutorials for TensorFlow assume the good understanding of ML, Linear Algebra, and concept like numerical instability, cross entropy, logits are rearly well explaind.(A flawed exception to that rule is this udacity course). But learning doesn’t work like that it is always a process where you create a link in your brain between something that you know and something that you want to learn. You can notice a ? in place where the number of tutorials that I plan to write but the truth is I don’t know yet :)Tensor Algebra and Tensor Calculues have a lot of appliation and ML is only one of them. I this coures I will use the simplest possible examples from the real world to teach you, my dear reader, and myself about the basics and more advanced uses of TensorFlow. We will lern how mayans encrypted their messages, how to calculate ange between two vectros, how to transform one figure into another, reduce the size of an image, if i’m capable of understanding it first: solving Enstein field equasions. All using tensorflow. On examples that have nothing to do with machine lerning I will cover what is error fucntion, softamx function, cross entropy and many others. After we get a very good grasp on theas concepts we will move and do some supervised learning (one of the branch of machine lerning). I dedicated this series to programers comming from the OO world and so thouse concepts that might be unfamilliar will be exaplained in paifull details, others like loops, if-statemnts won’t be mentiond.

Start here if you don’t care why Ido it :)

With that probably overly long intro let’s start. First environment: I will use google Colaboratory it is on VM environemt that runs in the cloud. You can save you code on you google drive. But if you are swept by all the privacy scandals and deleted your gmail account you can install anaconda3 env on your laptop.

Then copy the code above into the editor and run. The output should look like this:

initial: [16, 18, 5, 16, 1, 18, 5, 27, 20, 15, 27, 14, 5, 7, 15, 20, 9, 1, 20, 5, 0] encrypted version: [-189. -194. -96. -133. -125. -89. -36. 42. 40. 24. 28. 34. 10. 7. 205. 212. 101. 149. 126. 107. 41.]

Now for the explanation, and this is a little bit outdated (eager execution will be covert in one of the next posts), TensorFlow programs are created in two stages first you write a bunch of code that construct an execution graph that part cost You a lot an a CPU a little. And in the next you execute by feeding input nodes of that graph some data. The reason for that is that it allows TensorFlow to optimize execution put some work on the GPU some on TPU etc. In the above example everything between lines 9 and 18 is responsible for constructing a graph. And everything from to 22 to 24 for execution.

(9) t_input = tf.placeholder(tf.float32)

(11) t_key = tf.constant([[-3, -3, -4], [0, 1, 1], [4, 3, 4]], tf.float32)

In line 9 and 11 we define a tf.placeholder and a tf.constant. A place holder is like a entry point in our graph. To that node we are going to feed the data when we are going to run the graph. Constant is a tensor which value do not change. In both those declarations we also had to pass a type in our case it is tf.float32 meaning 32-bit floating point.

(15) t_reshaped = tf.reshape(t_input, [3, -1]);

In line 15 we do our first transformation we reshape our tensor from having 1 row to having 3 rows. The first parameter is our input, the second is the shape of a desired array -1 in the place where we give the number of columns tell us that we dont care how many column will the output array have. This is a desired output if this the end of our graph (In line 5 of our code we have the original declaration):

t_encrypted_rolled = tf.matmul(t_key, t_reshaped);

The reason we have to do the reshaping is the next operation. Here we do actual ecryption we take the t_key, our encryption matrix, and multiplying it by our reshaped t_reshaped input array. Tensor mutiplication demands that the number of column in the first matrix must mach number of rows in the second matrix.

The multiplication and the result are seen on the left.

t_encrypted = tf.squeeze(tf.reshape(t_encrypted_rolled , [1, -1]));

And the last line of your graph declaration is actually a nested instruction. Here we are doing two things first we are using the tf.reshape to go back to one dimensional tensor but that operation actually that operation leaves us with the 2-D tensor it just one is empty. To fix that we use tf.squeeze to reduce the dimensionality (As a homework try to remove the squeeze operation you will see [[…]] in the output instead of […]).

encrypted = sess.run(t_encrypted, feed_dict={t_input:unencrypted})

Now we run our graph. First parameter is the node we want to evaluate. In our case it is the last node t_encrypted, but there is nothing stopping us from putting any other node from our graph (this is the second homework try putting different nodes there see what happends). Second parameter is the input data in this case it’s a numpy array of numbers that we want to hide.

Now so far we only encrypted the numbers, hide them from our enemies. For the third homework you have to write the program that will do the decryption. Assuming that both parties have the encryption matrix saved (In my code it is: t_key constant). And you have to do that:

First argument is the decoding matrix that is a result of inverting an encoding matrix (look for inverting operation in TensorFlow documentations). The second argument is our encrypted matrix of numbers folded to three rows. Good luck.