Master Neural Networks: The Perceptron

Learning Perceptrons: A Beginner’s Guide to Neural Networks with JavaScript

The Perceptron is one of the simplest types of artificial neural networks, and it’s the foundation upon which more complex neural networks are built. In this article, we’ll walk through a basic implementation of a Perceptron in JavaScript, explaining each step in plain language to help you grasp the fundamental concepts. By the end of this post, you’ll have a solid understanding of how a Perceptron works and how it can be used to classify data.

Resources First!

Complete Perceptron Code: https://github.com/Jerga99/neural-network-course/blob/main/perceptron.js

Full Neural Networks Course: https://academy.eincode.com/courses/master-neural-networks-build-with-javascript-and-react

What is a Perceptron?

At its core, a Perceptron is a simple algorithm that can make predictions based on input data. It’s used for binary classification, meaning it can decide between two classes, like yes/no, true/false, or in our case, whether an object is more like a pencil or an eraser.

The Dataset

Let’s start with the data. We have two sets of data:

• Training Data: This is the data we use to train our Perceptron. It consists of inputs (features) and labels (the correct answer for each input).
• Test Data: After training, we use this data to see how well our Perceptron can predict new, unseen inputs.

Here’s the data we’re using:

// Train Inputs [weight(g), scale size (1–10)]
const trainInputs = [
 [2, 7],
 [3, 6],
 [1, 1],
 [1, 2],
 [2, 1]
];

// Train Labels
const trainLabels = [1,1,0,0,0];
const testInputs = [
 [2, 6], // pencil-like
 [3, 7], // pencil-like
 [1, 3], // eraser-like
 [2, 2], // eraser-like
 [2, 5] // pencil-like
];
// Test Labels
const testLabels = [1, 1, 0, 0, 1];

• Inputs: Each input is a pair of numbers representing an object’s weight (in grams) and its size on a scale from 1 to 10.
• Labels: Each label is either 1 (pencil-like) or 0 (eraser-like).

The Perceptron Model

The Perceptron algorithm works by calculating a weighted sum of the input features and then applying an activation function to decide the output. Here’s the basic structure of our Perceptron class:

class Perceptron {
 constructor(learningRate = 0.01) {
 this.weights = Array(2).fill(0).map(() => Math.random() * 0.3–0.1);
 this.bias = 0;
 this.learningRate = learningRate;
 }

activationFunction(sum) {
 return sum > 0 ? 1 : 0;
 }
}

• Weights: These are numbers that determine how important each input feature is.
• Bias: This is an extra parameter that helps the Perceptron model data that isn’t perfectly centered around the origin.
• Learning Rate: This controls how much we adjust the weights in each training step.

The Training Process

Training the Perceptron involves the following steps:

1. Calculate the Weighted Sum: For each input, we multiply each feature by its corresponding weight and add them up. Then, we add the bias to this sum.

let sum = this.bias;
for (let i = 0; i < inputs.length; i++) {
  sum += inputs[i] * this.weights[i];
}

2. Apply the Activation Function: The activation function checks if the weighted sum is greater than 0. If it is, the Perceptron predicts 1, otherwise it predicts 0.

const prediction = this.activationFunction(sum);

3. Update Weights and Bias: If the prediction doesn’t match the actual label, we update the weights and bias to make the Perceptron better at predicting next time.

if (label != prediction) {
  for (let i = 0; i < this.weights.length; i++) {
    this.weights[i] += this.learningRate * (label - prediction) * inputs[i];
  }
  this.bias += this.learningRate * (label - prediction);
}

Training the Perceptron

We train the Perceptron over several epochs. An epoch means going through the entire training dataset once. Here’s the training loop:

const EPOCHS = 3;
const perceptron = new Perceptron(0.01);

for (let epoch = 0; epoch < EPOCHS; epoch++) {
  perceptron.train(trainInputs, trainLabels);

  // Calculate accuracies
  const trainingAccuracy = calculateAccuracy(trainInputs, trainLabels);
  const testingAccuracy = calculateAccuracy(testInputs, testLabels);

  console.log(`Epoch ${epoch + 1}`);
  console.log(`Training Accuracy: ${trainingAccuracy}%`);
  console.log(`Testing Accuracy: ${testingAccuracy}%`);
  console.log("-------------------------------------")
}

• After each epoch, we calculate the accuracy on both the training data and the test data. This helps us understand how well the Perceptron is learning.

Results

Finally, we print out the weights and bias after training. These values show how the Perceptron is set up to make predictions:

console.log("weights: " + perceptron.weights);
console.log("bias: " + perceptron.bias);

Training and Testing Accuracy: What Do They Mean?

After training your Perceptron, you’ll calculate the accuracy on both the training and test datasets. Here’s what these metrics tell you:

• Training Accuracy: This measures how well your Perceptron performs on the training data it has seen before. High training accuracy indicates that your model has learned the patterns in the training data effectively.
• Testing Accuracy: This measures how well your Perceptron performs on new, unseen data (the test set). High testing accuracy suggests that your model can generalize well to new data, which is crucial for any machine learning model.

Overfitting vs. Underfitting:

• Overfitting: If your training accuracy is very high but your testing accuracy is low, your model may be overfitting. This means it has learned the training data too well, including noise or irrelevant patterns, and doesn’t perform well on new data.
• Underfitting: If both training and testing accuracy are low, your model might be underfitting, meaning it hasn’t learned the data well enough. This could be due to insufficient training or a model that’s too simple.

Conclusion

In this article, we walked through a simple implementation of a Perceptron in JavaScript. We started with a basic understanding of the Perceptron, explored the training process, and then examined the results. The Perceptron is a powerful yet simple tool for binary classification tasks, and understanding it lays the groundwork for more advanced machine learning models.

Feel free to experiment with the code, tweak the learning rate, or try different datasets. Understanding the basics is key to mastering more complex concepts in machine learning!

To learn more, check the ‘Resources’ section at the beginning of this article.

Cheers,

Filip