All Things JavaScript — Part III (Machine Learning using JavaScript!)

Please read part I and part II of this series before proceeding.

Reference — link

JavaScript is literally being used everywhere. I have stated this probably a gazillion times now. Sorry for that. Last time, we talked about how we could leverage Node.js runtime environment and powerful JavaScript frameworks like Electron.js to create super-awesome desktop applications. In this post, we shall talk about how we could create Machine Learning and Deep Learning models using JavaScript. But, before that, let’s answer the question —

Why JavaScript for Machine Learning and Deep Learning?

  1. No installations of specific packages required on client side.

Models created using native JavaScript would require just a browser to run and no extra drivers or packages/libraries would be needed.

Gone are the days of “pip install”.

pip install everything!

2. Interactive ML.

We can create awesome web applications by combining JavaScript based machine learning models with front-end frameworks like Angular and React.

Check out: Use your phone’s camera to identify emojis in the real world. Can you find all the emojis before time expires?

3. Access to online user activity.

We can create machine learning models that take advantage of user activity such as user clicks, user cursor movements, etc.

4. User data does not leave client side.

User data used for training and prediction purposes does not have to leave the client side. This ensures user data privacy.

5. Model once, run everywhere.

Since, JavaScript code can execute on any platform that supports an internet browser, machine learning models developed using JavaScript can be reused almost on every platform. It eliminates the need to model differently for mobile and desktop devices.

Enough with the reasons!

There are a number of JS libraries using which one can develop machine learning models. However, TensorFlow.js is the most popular.

TensorFlow.js

TensorFlow.js is a webGL accelerated, browser based JavaScript library for training and deploying Machine Learning and Deep Learning models. Using TensorFlow.js, we can do the following —

  • Develop Machine Learning models in the browser. TensorFlow.js provides flexible and intuitive APIs for building and training models.
  • Use pre-trained Machine Learning models and retrain them to fit user-specific data.
  • TensorFlow.js provides convertors which allows us to use existing TensorFlow models developed using Python.

One of the best demos that I found out on TensorFlow.js’s website was Webcam Controller.

Webcam Controller — Pacman

Implementing NOR gate Linear Classifier using TensorFlow.js —

Let’s get down to the implementation part. We are going to develop a Neural Network based Linear Classifier. We will be implementing a NOR gate linear classifier. Truth table for a NOR gate is as follows —

NOR gate truth table.

We will be trying to create a neural network which would output a value of 1 on an input of [0,0] and a value of 0 on an input of [0,1] or [1,0] or [1,1]. This is a linear classification problem. We can understand it by plotting these points on a 2d plane.

Scatter plot for the NOR data points.

The 3 data points [0,1], [1,0] and [1,1] belong to the same class and have been plotted with the colour blue as opposed to the data point [0,0] which is plotted with colour red. As inferred from the scatter plot, we can sense that the two classes are linearly separable and we can use a simple neural network model to find a linear line equation which can help us to accurately classify these data points. We will be creating a simple neural network having an input layer and an output layer as described in the below diagram —

Neural Network Architecture

To create our neural network model, we start by including TensorFlow.js script in our HTML document using the <script> tag.

<script src=”https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.9.0"></script>

We can now use TensorFlow.js to create our model. We start by creating tensorflow variables for input and actual output data, and setting up weights and biases of our model.

<!--We will store the losses for each epoch in this array-->
var losses = [];
var epochs = [];
<!--Creating tensor for input data.-->
const xs = tf.tensor2d([[0, 0], [0, 1], [1, 0], [1, 1]]);
<!--Creating tensor for output data-->
const ys = tf.tensor2d([[1], [0], [0], [0]]);
<!--Defining the weights and bias-->
const w = tf.variable(tf.randomNormal([2, 1], 0, 0.1));
  console.log("--------Initial-------");
w.print();
const b = tf.variable(tf.randomNormal([1]))
b.print();
console.log("--------Initial-------");

Next, we create a JavaScript function which will actually calculate the output of our neural network. Along, we that we also create a loss function.

<!--This is a output function, which takes in the input. Multiples the weights with the input and performs RELU on that-->
  const outputFunction = input => {
var weightInputMul = tf.matMul(input, w);
var biasAdded = tf.add(weightInputMul, b);
return tf.relu(biasAdded);
}
<!--This is a loss function which caluclates mean square error-->
  const lossFunction = (output, y) => output.sub(y).square().mean();

We now setup constants such as epochs and learning rate. Also, we train the model by applying a ‘Stochastic Gradient Descent’ optimiser.

const epoch = 1000;
const learningRate = 0.15;
<!--Setting up the optimiser and training the model-->
const optimizer = tf.train.sgd(learningRate);
for (let i = 0; i < epoch; i++) {
<!--dataSync() method pulls the latest values of the tensorflow variables from memory-->
      w.dataSync();
b.dataSync();
<!--Minimising the loss-->
optimizer.minimize(() => {
var loss = lossFunction(outputFunction(xs), ys);
loss.data().then(lossValue => {
epochs.push(i);
losses.push(lossValue);
})
return loss;
});
}

Once, the model is well-trained we can generate output by passing in values from our dataset. We use the following piece of code to generate output from our trained model.

const firstInput = 0;
const secondInput = 1;
const sampleInput = tf.tensor2d([[firstInput, secondInput]]);
const pred = outputFunction(sampleInput).dataSync();
console.log("The predicted output for input "+firstInput+" and "+secondInput+" is "+Number(pred));

Below is the loss function graph that I plotted using Graph.js.

Loss over 50 epochs

You can take a look at the experiment here. I have implemented the neural network we discussed along with a Chart.js visualisation of the loss function.

Demo

That’s it! We have successfully developed a neural network model using TensorFlow.js.

Full source code for the project is available here.

To know more about TensorFlow.js, I would recommend the official website.

You could also watch this awesome video from TensorFlow Dev Summit 2018.

If you are still here,

Sincerely Thanks!