MySQL handlersockets and ES2016 async/await.

Sergey Dolin
Jul 21, 2017 · 4 min read

While we were going to move our internal tracking service to the public service known now as Way.Today it was obvious we would have to improve the insert and update database operations performance greatly in order to handle lots of incoming messages from GPS trackers.

The expected solution of taking a noSQL database seemed to be natural choice but there were dragons of the backing up, replications, reporting tools. Of course, nothing was really unsolvable, but lacking the ability to issue a simple query against the data in few finger movements really disappointed me.

This was how MySQL handlersockets come to us. The ability to get best of both worlds was really promising and even shining and we got in.

As usual nothing was going easy and we faced a few problems with the keeping connections alive, ready to use indexes and sequences of the requests lead us to well know all nodejs developers “callbacks hell”. The mess of parentheses was so terrible we decided to move to ES2015 generators first and ES2016 async function next. The async functions win the generators in the grade the latter does not worth to be mentioned. In fact generators used for async coding appear to be not much better than the callbacks are, while async functions being nothing but a function returning promise showed themselves superb.

And now there are final handlersocket interface.

First, the common nodejs problem “to run few async services on startup sequentially”. It has extremely elegant solution with ES2016:

index.html

(async () => {
...
try {
log.info('Starting Way.Today...');
log.info('ZMQ Notification Starting...');
zmqNotifier = await zmqNotifierFactory();
log.info('Handlersocket Backend Starting...');
hsBackend = await hsBackendFactory(zmqNotifier);
...
log.info('Way.Today started.');
} catch (e) {
log.error(e);
...
if (hsBackend) hsBackend.close();
if (zmqNotifier) zmqNotifier.close();
}
})()

Looks very simple, even bore and pretty “magic” for the traditional nodejs developers like we were a year ago. To explain that is under the hood the code above may be rewritten in the equivalent “promises based” code:

...
log.info('Starting Way.Today...');
log.info('ZMQ Notification Starting...');

zmqNotifierFactory()
.then(notifier => {
zmqNotifier=notifier
log.info('Handlersocket Backend Starting...');
return hsBackendFactory(zmqNotifier);})
.then(backendFactory => {
... })
.then
...
.then(()=>{
log.info('Way.Today started.');
... })
.catch (e) {
log.error(e);
...
}

A bit curly is not it? But in fact, it is absolutely equal to the above in means nothing need to be changed in the definitions of zmqNotifierFactory and hsBackendFactory. The prettiest thing of async/await i love most is a function marked as to be “async” is nothing but a function returning promise and it looks as that for any “old-school” code. From the other hand, in order to “await” some function, it is not necessary to be declared as “async”, it is enough just to return promise. So, we can freely intermix modern await syntax with the old-fashioned “promises” functions.

// a function returning promise
const delay = (n) => {
const timestamp=new Date();
return new Promise(resolve=>{
setTimeout(()=>resolve(timestamp),n*1000)
})}
// use it with async/await environment(async ()=>{
const ts = await delay(5);
console.log(`The delay started at ${ts} completed`);
})()

…and vice verse:

// a function declared as async
const delay = async (n) => {
const timestamp=new Date();
// ... some async code with promises or awaits
return timestamp
}
delay.then( ts => {
console.log(`The delay started at ${ts} completed`);
});

The interesting thing here is if you return a result of async function you do not need to await it.

This would work:

function async asyncFunction(){...};function async returningResultFunction(){
...
return await asyncFunction()
}
const finalResult = await returningResultFunction();

but there’s no need to await the result — it gives us nothing but extra “.then” in the chain. The better rewritting of the same code is:

function async asyncFunction(){...};function async returningResultFunction(){
...
// no await here, just return a promise ...
return asyncFunction()
}
// ...and let it to be waited on outer level
const finalResult = await returningResultFunction();

Now, as we get familiar with the async/await lets look how it is used with handlersocket coding.

In order to get access to the MySQL through handler socket connections we need to get obects called “indexes”. To get them in every business logic function would be pretty tedious, so we introduced a simple wrapper:

const withIndexes = async (queryThunk) => {
if (!CONNECTION) await openConnection();
return queryThunk({lastId: LAST_ID_INDEX, spareIds: SPARE_IDS_INDEX,...});
};

The wrpapped first check the connection has not been closed (either because of timeout or the mysql served was gone) and reopen it if it has been. Opening the connection will initialize global variables with the indexes which we are passing within a wrapped function via an object attributes.

Now the business logic function may be implemented as:

const getId = async () => withIndexes(
async ({lastId, spareIds}) => {
log.debug('db-hs.getId');
// lastId has callback interface, so let's wrap it in
// promise and await the promise
const lr = await new Promise((resolve, reject) => lastId.find('>', [0], function (err, lr) { if (err) return reject(err); resolve(lr); }));// ...some bore code to calculate has been skippedreturn id;
}
}
)

So we defined async function which can be awaited in the outer code and avoided callback hell with the awaiting promises in the inner code.

Now let’s see the openConnectin function implementation full of async calls but looking as it would be sequential:

const openConnection = async () => {//wrap callbacked "connect" function in Promise and "await" itCONNECTION = await new Promise((resolve, reject) => {
hs.connect({host: config.hsHost, port: config.hsPort}, function () {
resolve(this);
})
.on('error', function (err) {
reject(err);
})
.on('close', function () {
CONNECTION = null;
});
});
CONNECTION.on('error', function (err) {
CONNECTION = null;
});
// the same with "openIndex" functionLAST_ID_INDEX = await new Promise((resolve, reject) => {
CONNECTION.openIndex(config.hsDatabase, 'last_id', 'PRIMARY', ['id'], function (err, index) {
// TODO: set CONNECTION to null
if (err) return reject(err);
resolve(index);
});
});
// ... continue openning indexes
// ... and return a value to be either returned as a result
// or passed as a parameter to .then success function
return CONNECTION;
};

Imagine how happy we were discovering the new ES2016 world. Finally, coding with nodes stopped to be the fighting and become a pleasure. We never return back to the days of callbacks.

)
Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade