Look, no hands — Logistic Regression Classification in plain ole Javascript

Logistic Regression from scratch, with a dashboard

[Post is also available at quaintitative.com]

I thought it would be fun to try to code a logistic regression model in plain ole Javascript, with no machine learning libraries this morning.

I somehow went down a rabbit-hole with the visualisation though, and ended up doing a dashboard in p5.js to visualise the data generation, training and prediction process straight in the client/browser.

Before we go through the code, you can play around with the dashboard here.

A quick orientation:

• The buttons at the top allow one to first generate random data (x1 and x2 coordinates, classes y = 0 or 1), train a logistic regression model on the data, and then use the trained model to predict the class of a red dot in the chart that can be moved around.
• ‘Total Points’ allows the user to specify the total number of random data points to generate; and ‘Segment Class’ allows the user to specify the number of points in one of the classes (i.e. the difference will be in another class).
• ‘PointX’ and ‘PointY’ allow you to move the red dot in the right hand chart. The coordinates should range from 0 to 500.
• Next we have ‘Training Accuracy’, which will be updated when training is completed, and the ‘Logistic Equation’ used here.
• The chart on the left plots out the points with the different fill colors based on the class assigned; and the chart on right plots out the training accuracy over 500 epochs (in white) and the shape of the logistic curve (in blue).

The part of the code relating to the logistic regression model is the focus of this post, so I will zoom in on that. The visualisation is done using the p5.js library (instead of d3.js just for a change).

First, the random data is generated with a a simple loop and p5.js’ random function.

function generateData(total,boundary,upper,lower) {
    for (var i=0; i<total; i++){
        var X1, X2, Y;

        var cutoff = P5.random()*(upper-lower)+lower;

        if (i<=boundary){
            X1 = P5.random(lower,cutoff*0.9);
            X2 = P5.random(lower,cutoff*0.9);
            Y = 0;
        } else if (i>boundary){
            X1 = P5.random(cutoff*1.1,upper);
            X2 = P5.random(cutoff*1.1,upper);
            Y = 1;
        }

        data.push({x1:X1, x2:X2, y:Y});

    }

    var progresspercent = 0;

        $('.progress-bar').css('width', progresspercent+'%').attr('aria-valuenow', progresspercent);
    // console.log(data);
}

We then write the function for training.

function train(epochs, alpha){

    console.log(data);

    errors =[];
    A = 0.0;
    B = 0.0;
    C = 0.0;

    var count =0;
    for (var i=0; i<epochs; i++){
        var error;
        
        data.forEach(d=>{

            var predY;
            var func;
            func = A*d.x1/100+B*d.x2/100+C;
            predY = 1/(1+Math.exp(-func));
            error = predY - d.y;
            tempA = A;
            tempB = B;
            tempC = C;

            A = tempA + alpha*-error*predY*(1-predY)*d.x1/100;
            B = tempB + alpha*-error*predY*(1-predY)*d.x2/100;
            C = tempC + alpha*-error*predY*(1-predY)*1.0;
            
            // errors.push({error:error, iteration:count});
            // count++;
        })

        console.log('A', A, 'B', B, 'C', C);
        console.log('Error', error);
        errors.push({error:error, epoch:i});

        var accuracy = 1+Math.round(error*100)/100;
        $('#accuracy').text(accuracy);


        var progresspercent = 100*i/500;

        $('.progress-bar').css('width', progresspercent+'%').attr('aria-valuenow', progresspercent);
    }

    console.log(errors);
    

}

It’s pretty straight-forward. We declare 3 variables A, B, and C. Then we loop through the dataset, insert the x1 and x2 values in the logistic regression equation, get the predicted value and compare against the actual y. We then use that to update A, B, and C, and keep track of the errors at each epoch.

We also compute the accuracy shown in the browser and the progress bar.

The final piece is the predict function.

function predict(x1,x2){

    var predY;
    var func;
    var out;
    func = A*x1/100+B*x2/100+C;
    predY = 1/(1+Math.exp(-func));

    if(predY>0.5){out=1}
    else if (predY<0.5){out=0};

    return out;
}

We are basically plugging in the final A, B, and C values into the logistic regression equation and using it to predict the class of a point.

And that’s it. The full code is available here.

playgrd.com || facebook.com/playgrdstar || instagram.com/playgrdstar/