Facenet on Mobile — Part 3

Understanding it, a bit more …

Help out the underdog! Beat the Cybercrime learn how.

If you have not read my earlier stories about FaceNet Architecture and converting .pb to .tflite, i would suggest going through part-1 and part-2.

Let us take it forward from where we left in part-2. We converted Facenet checkpoint to Facenet frozen model .pb having just inference branch in it and striping out phase_train from the model. To verify this fact please use Tensorflow graph_transforms tool, as shown in the image below:

Similarly, we have awesome tool for .tflite models too. I would highly encourage to use Pete’s tflite_analyser tool (Thanks to Pete!). Please find the snippet of the output below:

Now, if you compare the output of these two model’s ‘weight size’, its quite the same.

23M weights x 4 (float datatype) = 93M bytes of weights.

You will also notice that complete models size for both .pb and .tflite is also around 93M.

Its time to verify if they spit out same embeddings for a given ‘image’?

Please note the model used in this article is NOT quantized yet, because post quntization we expect some degradation. But at this point we want to examine, if there is any change converting .pb to .tflite? Because this conversion is essential mere conversion of format, i.e. converting protocol buffers to flatbuffers.

This can be archived using following command:

$ tflite_convert --output_file model_mobile/my_facenet.tflite --graph_def_file facenet_frozen.pb --input_arrays “input” --input_shapes “1,160,160,3” --output_arrays "embeddings" --output_format TFLITE

Theoretically, both models should spit out same embeddings for a given image, let us test it:

Protocol Buffer — Model

The below script take .pb model and an image of mine to output 128-bytes embedding. Here, we are normalizing the image using prewhiten().

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import tensorflow as tf
import numpy as np
import facenet
from skimage import io
def prewhiten(x):
  try:
    x = io.imread(x)
  except:
    print('Image is corrupt!')
    return 0
  mean = np.mean(x)
  std = np.std(x)
  std_adj = np.maximum(std, 1.0/np.sqrt(x.size))
  y = np.multiply(np.subtract(x, mean), 1/std_adj)
  return y
def main():
  with tf.Graph().as_default():
    with tf.Session() as sess:
      np.random.seed(seed=666)
# Load the model
      print('Loading feature extraction model')
      facenet.load_model('facenet_frozen.pb')
# Get input and output tensors
      images_placeholder = tf.get_default_graph().get_tensor_by_name("input:0")
      embeddings = tf.get_default_graph().get_tensor_by_name("embeddings:0")
      embedding_size = embeddings.get_shape()[1]
# Run forward pass to calculate embeddings
      input_img = 'Milind-Deore.jpeg'
      image = prewhiten(input_img)
input_data = np.zeros((1, 160, 160, 3))
      input_data[0,:,:,:] = image
      input_data = input_data.astype(np.float32)
      feed_dict = { images_placeholder:input_data }
      emb_array = np.zeros((1, 128))
      emb_array = sess.run(embeddings, feed_dict=feed_dict)
print('EMB/Label : \n', emb_array[0])
if __name__ == '__main__':
     main()

It outputs something like below : (minuscule snippet)

FlatBuffer — Model

Similarly for .tflite model, below script take a same image of mine and output 128-bytes embedding.

import numpy as np
import tensorflow as tf
from skimage import io
# Normalize the image
def prewhiten(x):
     try:
         x = io.imread(x)
     except:
         print('Image is corrupt!')
         return 0
     mean = np.mean(x)
     std = np.std(x)
     std_adj = np.maximum(std, 1.0/np.sqrt(x.size))
     y = np.multiply(np.subtract(x, mean), 1/std_adj)
     return y
# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path='my_facenet.tflite')
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Test model on random input data.
input_img = 'Milind-Deore.jpeg'
image = prewhiten(input_img)
input_data = np.zeros((1, 160, 160, 3))
input_data[0,:,:,:] = image
input_data = input_data.astype(np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print('INPUTS: ')
print(input_details)
print('OUTPUTS: ')
print(output_details)

print('--------- Output Data --------------')
print(output_data)

The output snippet is as below:

Good News! The embeddings looks similar.

Can you now try applying quantization QUANTIZED_UINT8 and see the accuracy change, because then comparing embeddings won’t make any sense as the quantization would convert the output embeddings fromfloat32 to int8datatype.

Citation:

• Thanks to Pete Blacker for providing awesome tflite analyser.

You can connect me on | LinkedIn | Website | Github |