Perfect-TensorFlow: Write Your Own AI Server in Swift

Published in

Server Side Swift and More

6 min readJun 30, 2017

TensorFlow is one of the most popular Deep Learning frameworks in the world. Now it has new language binds: Perfect-TensorFlow, an Open Sourced Swift library contributed by PerfectlySoft Inc. If you are a Swift developer, this new library provides you a good approach to add more value to your app by incorporating a backend server to perform machine learning tasks. Fortunately, it is easy to learn because of its simple API.

Introduction

Workflow below shows how the Perfect-TensorFlow classes work:

TFLib: TensorFlow C API DLL Low levelled library
  |
TensorFlow: Runtime Library
  |
  |------ Shape: dimension info of a tensor
  |         |
  |       Tensor: multi-dimensional array
  |         |
 Graph -- OperationBuilder: construct an operation from tensors
  |         |
  |       Operation: a computation on tensors
  |         |
  |       Output: a symbol of a tensor produced by an Operation
  |         |
  |------ Session: driver for graph execution.
            |
          Runner: Run Operations and evaluate tensors

The programming concepts you need to know are:

Tensors are multi-dimensional arrays, and Shape is an alias of dimension.
Different Tensors can setup into Operations, which each of them can be constructed by a OperationBuilder.
A Graph is a data flow representing a total computation of all Operations.
A Session Runner is the actual action of the Graph computation, i.e, place the inputs, run the math then eject out Output tensors.

The Simplest Workflow

There are three key steps to construct an AI session:

Setup data input. Yes, you can put any data, theoretically, like numbers, texts, voices, images or even a 3D movie, as input;
Configure the graph, what we called Model. This is the hardest and most tricky part but luckily, a lot of data scientists have already contributed many pre-trained models available online. Yes, the good news is you can load these TensorFlow models into your graph whenever you want.
Run the Session and get the prediction for your data. For example, if your input is an image, then the prediction may be identification of all objects listed in this image; Or, if the input is a voice clip, then the prediction may be who is talking about what, or even more with a power of text prediction — what is the speaker going to say in the next minute?

data input — → Graph (load from model) — -> prediction

Quick Start

Before programming, I strongly recommend to install the latest version of Swift, so you may also need Xcode if possible. Perfect-TensorFlow has already placed an installation script of TensorFlow runtime library, “libtensorflow.so” of CPU version “install.sh” to simplify the process. For more installation options, such as GPU/CPU and multiple versions on the same machine, please check TensorFlow website: “Installing TensorFlow for C” (https://www.tensorflow.org/install/install_c)

Perfect TensorFlow Application

To use this library, add dependencies to your project’s Package.swift with the LATEST RELEASE:

.Package(url: "https://github.com/PerfectlySoft/Perfect-TensorFlow.git", majorVersion: 1)

Then declare the library:

// TensorFlowAPI may be useful in low-level operations
import TensorFlowAPI

// This is the Swift version of TensorFlow classes and objects
import PerfectTensorFlow

// Making an alias of `TensorFlow` Class is a good idea:
public typealias TF = TensorFlow

Library Activation

⚠️NOTE⚠️ Prior to use ANY ACTUAL FUNCTIONS of the Perfect TensorFlow framework, call `TF.Open()` first:

// this action will load all api functions defined 
// in /usr/local/lib/libtensorflow.so
try TF.Open()

Please also note that you can load the library by a specific path which is useful when you want to keep different versions, or CPU/GPU libraries at the same time.

// this action will load the library with the path
try TF.Open("/path/to/DLL/of/libtensorflow.so")

“Hello, Perfect TensorFlow!”

Here is the Swift version of “Hello, TensorFlow!”, which has only one input in the whole graph so if run it, the output will be the input itself:

// define a string tensor
let tensor = try TF.Tensor.Scalar("Hi, Perfect TensorFlow! 🇨🇳🇨🇦")

// declare a new graph
let g = try TF.Graph()

// turn the tensor into an operation
let op = try g.const(tensor: tensor, name: "hello")

// run a session
let o = try g.runner().fetch(op).addTarget(op).run()

// decode the result      
let decoded = try TF.Decode(strings: o[0].data, count: 1)

// check the result
let s2 = decoded[0].string
print(s2)

Matrix Operations

To better express the relationships among tensors, operations, and graph, let’s take a look at how to use a graph to describe a matrix multiplication.

In this example, we put two matrices, A and B, as input. The operation is to call A x B (transpose matrix of B) and after the computation, the session should generate an output tensor:

/* Matrix Muliply:
| 1 2 |   |0 1|   |0 1|
| 3 4 | * |0 0| = |0 3|
*/
// input the matrix.
let tA = try TF.Tensor.Matrix([[1, 2], [3, 4]])
let tB = try TF.Tensor.Matrix([[0, 0], [1, 0]])

// adding tensors to graph
let g = try TF.Graph()
let A = try g.const(tensor: tA, name: "Const_0")
let B = try g.const(tensor: tB, name: "Const_1")

// define matrix multiply operation
let v = try g.matMul(l: A, r: B, name: "v", transposeB: true)

// run the session
let o = try g.runner().fetch(v).addTarget(v).run()
let m:[Float] = try o[0].asArray()
print(m)
// m shall be [0, 1, 0, 3]

Load a Pre-trained Model

There are two different approaches to loading pre-trained models.You can either load a pre-defined graph as a GraphDef Protobuf which can be read/write from a binary file,

let g = try TF.Graph()
try g.import(definition: def)

or generate a new session from a pre-fabricated model.

let g = try TF.Graph()

// the meta signature info defined in a saved model
let metaBuf = try TF.Buffer()

// load the session
let session = try g.load(
	exportDir: "/path/to/saved/model",
	tags: ["tag1", "tag2", ...],
	metaGraphDef: metaBuf)

You can try them both, depends on what kind of models in practice.

As A Server

With Perfect, one of the most popular and the most powerful HTTP Server Frameworks in Swift, Perfect-TensorFlow can help you build your own AI servers in minutes. That’s the idea:

Use Perfect as a server.
Allow incoming inputs from browsers or your mobile apps as web request.
Use Perfect TensorFlow to analyze your input.
Respond to the web request with a prediction based on your input.

The benefit of making AI server is obviously significant:

Making more complicated predictions by much larger AI models.
You can improve it from the backend without updating your mobile apps.

Of course you can run the same computation on your mobile device, however, servers can do it much FASTER, and can save your palm from over burn — I am not kidding because AI requires much more resources such as CPU or memory, so don’t be surprised if your mobile got overloaded.

You can find a good example of how TensorFlow works with Perfect Servers: Perfect TensorFlow Demo. In this example, you can drag & drop any local image files or draw a scribble online to test if the server can recognize the picture content.