Part 2 : Testing Queue Worker

Using Node Js and Beanstalk on Ubuntu 18.04

Photo by Sarah Dorweiler on Unsplash

As my previous article, I want to try using Beanstalk as queue worker for my application. Previously, I am using PHP Laravel as my tech stack, for now, I want to try using Node JS, if possible maybe express js. The result / tutorial is explained below.

Beanstalkd

Actually I already explain how to install & configure beanstalkd in the previous article, kindly skip this step if you already install one.

If you not install yet, here is the steps.

• Installation

sudo apt-get install beanstalkd

• Start the service

sudo systemctl start beanstalkd 
sudo systemctl enable beanstalkd 
sudo systemctl status beanstalkd

• Testing installation

By default, beanstalk using port 11300, we can test it using telnet.

telnet localhost 11300 #for access the beanstalk
list-tubes #try this command to see list of tasks

This is the result example :

• Exit

quit

Node JS

There’s many tutorial on installing node js, but I will give the steps here.

• Download installation

You can access https://nodejs.org/en/download/ to download newest node js version for Windows, Mac or Linux.

In my case, I will install in Ubuntu 18.04, so I will use NodeSources installation. I will use node version 12, so the package name should be setup_12.x

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -

• Install — just follow the installation steps if any—

sudo apt install nodejs

• Test your installation

Run these commands to test your installation.

Testing node js installation result

• Create a project

Through command prompt, access the directory that you want, and run these commands :

mkdir your-project-name
cd your-project-name
npm init // then, follow the steps

Beanstalk Client for Node JS

As explained on the Beanstalkd wiki, there’s many beanstalk client for node js. I personally using nodestalker.

Beanstalk client for node js

Here’s a few steps using nodestalker :

• Installation

As explained in the nodestalker github page, just run this command in your nodejs application

npm install nodestalker

• Testing installation

First, create aindex.js file in your node application, then add this code (it from the nodestalker github page)

var bs = require('nodestalker'),
    client = bs.Client('127.0.0.1:11300');

client.use('default').onSuccess(function(data) {
  console.log(data);

  client.put('my job').onSuccess(function(data) {
	console.log(data);
	client.disconnect();
  });
});

After that, run node index.js , if it successfully running, the result should be more or less similar with this.

Testing result

Testing the Application

For simple usage, I will use the example provided in the nodestalker github page. It will add a job, receive the job synchronously, and receive the asynchronously.

• Add a folder called jobs in your application and go inside the folder
• Create addjob.js — this file will add a single job —

var bs = require('nodestalker'),
    client = bs.Client('127.0.0.1:11300');

client.use('default').onSuccess(function(data) {
  console.log(data);

  client.put('my job').onSuccess(function(data) {
	console.log(data);
	client.disconnect();
  });
});

• Create handlejobs.js — this file will handle all the jobs —

var bs = require('nodestalker'),
    client = bs.Client('127.0.0.1:11300'),
    tube = 'test_tube';

client.watch(tube).onSuccess(function(data) {
    function resJob() {
        client.reserve().onSuccess(function(job) {
            console.log('reserved', job);

            client.deleteJob(job.id).onSuccess(function(del_msg) {
                console.log('deleted', job);
                console.log('message', del_msg);
                resJob();
            });
        });
    }

    resJob();
});

• Create handlejobsasync.js — this file handle all the jobs asynchronously —

var bs = require('nodestalker'),
    tube = 'test_tube';

function processJob(job, callback) {
    // doing something really expensive
    console.log('processing...');
    setTimeout(function() {
        callback();
    }, 1000);
}

function resJob() {
    var client = bs.Client('127.0.0.1:11300');

    client.watch(tube).onSuccess(function(data) {
        client.reserve().onSuccess(function(job) {
            console.log('received job:', job);
            resJob();

            processJob(job, function() {
                client.deleteJob(job.id).onSuccess(function(del_msg) {
                    console.log('deleted', job);
                    console.log(del_msg);
                    client.disconnect();
                });
                console.log('processed', job);
            });
        });
    });
}

resJob();

• Running the application

1. Add some jobs by running node addjob.js , you can run this script multiple time to create multiple jobs
2. Handle the existing jobs using node handlejobs.js or node handlejobsasync.js to handle it asynchronously

This is the my result :

Testing result

Final thought

The implementation is quite simple by using nodestalker. You can try using other beanstalk clients that I described. So, for next steps, I will like to combine my Laravel app and my node js app, maybe something like creating a job in Laravel and the job will be handled by Node js app or the other way around.

Best regards,

I Gusti Ngurah Arya Bawanta