How to run Tensorflow.js on a serverless platform : deploying models

This is the last part of a 3 articles serie.

In the first part, we introduced neural networks and TensorFlow framework basics.

In the second part, we explained how to convert existing models from Python to TensorFlow.js

Finally, we present, through an example, how to use an online TensorFlow.js model and deploy it rapidly using our WarpJS JavaScript Serverless Function-as-a-Service (FaaS).

Using an online model with TensorFlow.js

Many public models can be retrieved from web databases.

We’ll use the “toxicity” pre-trained model in the next sections as an example.

The toxicity model detects whether text contains toxic content such as threatening language, insults, obscenities, identity-based hate, or sexually explicit language. The model was trained on the civil comments’ dataset: https://figshare.com/articles/data_json/7376747 which contains ~2 million comments labeled for toxicity. The model is built on top of the Universal Sentence Encoder (Cer et al., 2018).

Browser-based usage:

The model can be directly loaded for use in JavaScript at:

https://cdn.jsdelivr.net/npm/@tensorflow-models/toxicity

In the html, add:

Then, in the JS code:

Node-based usage:

Toxicity is also available as a NPM module for Node.js (package that actually loads the model from the storage link above):

Then, in the JS code:

Deploying a model with WarpJS

As discussed before, inference on big data sets in the browser comes rapidly short in terms of performance due to model and data loading time as well as computing capabilities (even with accelerated backends).

Node.js allows to push further the performance limit by deploying on a high performance GPU engine and in the network neighborhood of the dataset, but the user will face complexity when trying to address distributed processing for the next performance step.

The WarpJS JavaScript FaaS enables easy serverless process distribution with very little development effort.

Example: toxicity model serverless deployment

WarpJS installation guidelines can be found here: Getting started with WarpJS

You can request WarpJS account here.

This article also provides a good tutorial on all steps to operate WarpJS.

In our WarpJS serverless operation, the browser acts as the primary input/output interface, through an index.html file.

It contains a text box to submit the input text to be analyzed and a “classify” button triggering the inference process.

WarpJS is a function-as-a-service platform for JavaScript.

Instead of creating HTTP endpoints and use HTTP calls to do a remote inference, we just have to build a client for this inference function, deploy it on its FaaS, import it in the main application (via import statement in index.js or via script tag in html) and then call it like any JavaScript function.

So in our case, the classify function will be run on the WarpJS FaaS.

index.js (code to be deployed on WarpJS FaaS, prediction function is highlighted in bold) :

index.js (main, browser-based application):

Deploying to the WarpJS FaaS is straightforward, just use “npm run deploy” to get the url of the deployed site and start playing with TensorFlow.js.

Feel free to access https://warpjs-744h4bixx1x93pg3oxc3hr4cf.storage.googleapis.com/index.html url to see the demo in action.

#MadeWithTFJS

Thanks!

About the author

Dominique d’Inverno holds a MSC in telecommunications engineering. After 20 years of experience including embedded electronics design, mobile computing systems architecture and mathematical modeling, he joined ScaleDynamics team in 2018 as AI and algorithm development engineer.