What Does the Event Loop Sound Like?
A musical journey inside Node internals
There’s a notion that coding and debugging are purely intellectual tasks. Car mechanics can determine engine issues by listening to the engine, scrutinizing its starting sound and reverberations like a physician might with their stethoscope. Meanwhile, developers are stuck with the Terminal Bell — ASCII code 7 — a relic from electronic typewriters. There’s much more than meets the eye.
This lack of audible feedback confines our intuition. Until very recently we had sounds: the phone tones of dial-up internet, the hard drive’s rpm and metallic needles, and the fans frenetically spinning from a heavy computational load. One could “sense” what was going on.
The fate of computer noises is akin to the electric car, where progress and clean design has made things slick to non-existent, leaving only silence, polished glass, and a shiny metal exterior. Meanwhile, carmakers are hiring fancy sound designers to “skeuomorphise” the sound of an accelerating engine, giving a sense thrust to the driver, and hopefully warn distracted pedestrians.
Developers should have the sense that “something’s off” with their runtime. This could also aid visually impaired developers limited by low information bandwidth through their eyes.
I don’t mean annoying beeps; I’m talking about beautifully designed material sounds. As if the innards of software were subtly metallic, glassy, or even made of wood and granite, depending on your theme. Themes could also range harmonically, ranging from atonal percussive sounds to chromatic, varying in keys and moods.
I envision these sounds as non-intrusive background elements, that is, not interfering with your music playlist. I wouldn’t say pleasant; that depends on your app’s performance.
Method
For my project, I’ll try to submerge myself into Node’s event loop and give it some physicality. I’ll do this by setting up hooks that generate MIDI signals, and process the signals in real-time to produce sound.
I’m not a sound designer, but I do like to tinker with interesting software. I was recently inspired by Ableton Live and Max for Live. Live is a Digital Audio Workstation (DAW for short) crafted for live performances and popular with electronic musicians. Max, a “software for experimentation and invention,” is a visual programming language by Miller Puckette and later David Zicarelli (founder of Cycling ’74) for processing sound signals, dating back to 1986. A descendant of MUSIC-N, a program written by Max — hence the name — Mathews in 1957 at Bell Labs.
In Max, one works within a “patcher” window, which borrows its UX metaphors from “patching” modular synths together with cables. Beyond the metaphor, one can even extend Max with physical patching cables and components. The visual metaphors bind perfectly with object-oriented programming, where objects send each other messages, and the code looks like a its object UML representation. Moreover, Max is data-flow oriented, where the entire program is a directed graph from input to output.
Max-for-Live is an integration of Max into Ableton Live, so that your patches work natively in Live, and is included in Ableton Live Suite (and its generous 90-day trial). Max for Live has a broad community. The latest version introduced Node for Max, with which I can turn a Nodejs script into a Max object. Ableton and Cycling ’74 formed a partnership (an independently run entity, wholly owned by Ableton). An open source alternative, of which I’m less familiar is Pure Data (Pd), also developed by Max’s Miller Puckette.
Warning: when tinkering with sound, it’s very important to output things at safe levels to avoid damaging one’s speakers or worse — hearing.
Libuv
There’s no good way to hooking up to the event loop from Node itself. Yes, there are async-hooks and process._getActiveHandles(), but I want the real thing. Digging a little bit I found the dragon in the cave: libuv. Originally written as Unix-only Libev by Marc Lehmann, libuv is Nodejs’s cross-platform library providing an abstraction over I/O polling mechanisms.
Libuv is lean, performant, and has a stable API (same API for both Windows and Unix systems!). Besides Nodejs, Libuv churns under the hood of many other interesting projects.
I’ll be working with the Unix implementation. Besides being its own repo, Libuv resides inside Node, under deps/uv.
Libuv is asynchronous and event-driven. It polls for I/O through an event loop and schedules callbacks. This is pretty much the description of Nodejs, Libuv defined Node’s entire character since its inception.
In an event loop, an application registers a callback for an event and waits for that event to occur, while the loop repeats as long as there are active handles. This is very efficient for I/O scheduling, as it’s non-blocking and thus can handle many clients under few threads. It’s not the best paradigm for heavy computation, as they might block the event loop, and thus hang the thread.
The heart of the loop is the function uv_run inside core.c. The state of the loop is a struct passed around by reference, uv_loop_t (defined as uv_loop_s in the header file here). Here’s the while loop keeping node alive:
The above loop can be represented by the famous diagram from Nodejs documentation:
Note: I can’t just print the loop struct as we’d do in JavaScript (e.g.: console.log(loop)). This is due to the language’s lack of “reflection.” Reflection is the ability of a language to see its own structures. We need to print each field ourselves. To know which loop we’re running, we can just print its pointer address with the %p placeholder:
printf("uv_run loop: %p mode: %d\n", loop, mode);Let’s add the log lines (line 365 onward) and compile by running:
$ ./configure
$ make -j4The -j4 option will cause make to run 4 simultaneous compilation jobs which may reduce build time. It still takes around an hour.
Compiled
Once I have a node binary file I can run it with input and see the event loops running. Passing the -e (eval) flag we can just inline JavaScript as a string.
./node -e "console.log(\"hello node\")"Results in:
uv_run loop: 0x7fbdd780d8e8 mode: UV_RUN_DEFAULT
hello node
uv_run loop: 0x111dda800 mode: UV_RUN_DEFAULT
uv_run loop: 0x111dda800 mode: UV_RUN_ONCE
uv_run loop: 0x111dda800 mode: UV_RUN_ONCE
uv_run loop: 0x7fbdd780d208 mode: UV_RUN_ONCENote: I replaced the int values of the ‘mode’ enum for its name.
We can see the event loop churning. Nodejs uses the default loop as its event loop.
Handles
The next thing of interest are handles. These are the actual jobs keeping the loop alive. They can be of different types and are added to the queue inside the loop data structure. Its initialization function is defined in uv-common.h as uv__handle_init. This is actually not a C function but a Macro, code intended to be substituted in place by the compiler. I’ll add a log line in there, as well as in uv__handle_start and uv__handle_stop.
Now,
./node -e "console.log(\"hello node\")"outputs:
uv__handle_init 0x7ffa1100d528 0x7ffa1100d208 UV_SIGNAL
uv__handle_init 0x7ffa1100d2d0 0x7ffa1100d208 UV_ASYNC
uv__handle_start 0x7ffa1100d2d0
uv__handle_init 0x7ffa1100d708 0x7ffa1100d208 UV_ASYNC
uv__handle_start 0x7ffa1100d708
uv__handle_init 0x7ffa11808608 0x7ffa118082e8 UV_SIGNAL
uv__handle_init 0x7ffa118083b0 0x7ffa118082e8 UV_ASYNC
uv__handle_start 0x7ffa118083b0
uv__handle_init 0x7ffa11808718 0x7ffa118082e8 UV_ASYNC
uv__handle_start 0x7ffa11808718
uv_run 0x7ffa118082e8, mode UV_RUN_DEFAULT
uv__handle_init 0x10ef80b20 0x10ef80800 UV_SIGNAL
uv__handle_init 0x10ef808c8 0x10ef80800 UV_ASYNC
uv__handle_start 0x10ef808c8
uv__handle_init 0x7ffa10d087e0 0x10ef80800 UV_ASYNC
uv__handle_start 0x7ffa10d087e0
uv__handle_init 0x10ef76ab8 0x10ef80800 UV_ASYNC
uv__handle_start 0x10ef76ab8
uv__handle_init 0x7ffa01018130 0x10ef80800 UV_TIMER
uv__handle_init 0x7ffa010181c8 0x10ef80800 UV_CHECK
uv__handle_init 0x7ffa01018240 0x10ef80800 UV_IDLE
uv__handle_start 0x7ffa010181c8
uv__handle_init 0x7ffa010182b8 0x10ef80800 UV_PREPARE
uv__handle_init 0x7ffa01018330 0x10ef80800 UV_CHECK
uv__handle_init 0x7ffa010183a8 0x10ef80800 UV_ASYNC
uv__handle_start 0x7ffa010183a8
uv__handle_start 0x7ffa010182b8
uv__handle_start 0x7ffa01018330
uv__handle_init 0x7ffa02305b30 0x10ef80800 UV_TTY
uv__handle_init 0x7ffa024044c8 0x10ef80800 UV_SIGNAL
uv__handle_start 0x7ffa024044c8hello nodeuv_run 0x10ef80800, mode UV_RUN_DEFAULT
uv__handle_stop 0x7ffa02305b30 // UV_TTY
uv__handle_stop 0x7ffa024044c8 // UV_SIGNAL
uv__handle_stop 0x7ffa010181c8 // UV_CHECK
uv__handle_stop 0x7ffa010182b8 // UV_PREPARE
uv__handle_stop 0x7ffa01018330 // UV_CHECK
uv__handle_stop 0x7ffa010183a8 // UV_ASYNC
uv_run 0x10ef80800, mode UV_RUN_ONCE
uv__handle_stop 0x10ef76ab8 // UV_ASYNC
uv_run 0x10ef80800, mode UV_RUN_ONCE
uv__handle_stop 0x7ffa10d087e0 // UV_ASYNC
uv__handle_stop 0x7ffa11808718 // UV_ASYNC
uv__handle_stop 0x7ffa1100d708 // UV_ASYNC
uv_run 0x7ffa1100d208, mode UV_RUN_ONCE
This is pretty cool — we can even see a handle for “TTY,” which is the terminal handle used to print our console.log.
Trivia: TTY stands for teletype, a teleprinter company. This was the console to computers before monitors were a thing.
I could go on and print the requests, the smaller units of work in Libuv, but I think that would be too much at this point.
Parsing the logs into real-time MIDI signals
I made a small script that parses the terminal output of our modified Nodejs, and assigns them MIDI notes that I output in real time. MIDI is the standard digital interface for musical instruments, so this signal can also be played on your 90’s Casio keyboard. This was done in the following way:
- Inside a Node4Max script, spawn a
child_processwith our modified Nodejs executable, and assign handler functions to its events. Yes, we’re using a Nodejs thread to run and parse a separate Nodejs thread. The Nodejs running this script is a standard one that came bundled with Max-for-Live. - Create an rxjs
Subjectto which we can subscribe. This I will use as an abstraction over our event Stream. Eachdataevent will trigger anextevent to the subscriber. Additionally, since many logs are bundled together into one chunk, we will split them by the “newline” character and treat each separate line as a unique subscriber event. - Parse each one of these subscriber events, retrieving its type and address.
- Assign a MIDI note to each event by its type.
handle_startwill start the note (by assigning a “velocity” 100 to the note) andhandle_stopwill stop the note (by assigning a “velocity” 0 to the note). For simplicity, I decided that theintenum value of of the handle type will be its note (with100added to get a higher range of notes). SoUV_ASYNC, which has anenumvalue of1, will be note101(note F7, piano key 81), and so on. - In the script, we output the note to the
outletfunction frommax-api. This is a third party that shouldn’t be downloaded throughnpmnor be added to the dependencies inpackage.json.It is referenced by Node4Max when running it from inside Max-for-Live. I don’t like this design pattern, as it is coupled to the runtime and I need to mock it when running it outside Max.
6. Max for Live receives the MIDI signal, which we pass as a message to the native midiformat object. The original message consist of a tuple (an array) with note and velocity, which midiformat turns it into a valid MIDI signal that can be sent to midiout, the output of our patch.
7. Now we can use this patch as a MIDI effect in Ableton Live, which still requires a MIDI Instrument to transform MIDI signals into sound-waves that can be sent to your speakers. We add an instrument to the device chain (I selected a random instrument, “E-Piano Basic”), lower the Gain to a minimum just to be safe, start the script, and hear sound. This is the sound I got from a console.log:
This is what a console.log might have sounded like to Charles Babbage and Ada Lovelace as they cranked their Analytical Engine’s hand-wheel.
Let’s Make Some Noise
Express Server Initializing
This is a production-grade Nodejs Express server application starting up, connecting to a Mongodb database, opening a few UV_TCP and UV_IDLE handles and a loop_run waiting for incoming requests. Each type of handle has its own note.
Express Server Initializing (Port Busy Error)
This is a common error. Same server, but initialized unsuccessfully due to busy port error.
Error: listen EADDRINUSE: address already in use :::3007Nest Server Initializing
Same as above, similar sound signature.
Crypto: Diffie-Hellman Example
Example node code the crypto docs. I changed the instrument to a Synth with sustain to better capture the pending handle. UV_ASYNC, UV_PREPARE, and UV_CHECK ran for a few seconds before the event loop restarted.
const assert = require('assert');
const { createDiffieHellman } = require('crypto');// Generate Alice's keys...
const alice = createDiffieHellman(2048);
const aliceKey = alice.generateKeys();// Generate Bob's keys...
const bob = createDiffieHellman(alice.getPrime(), alice.getGenerator());
const bobKey = bob.generateKeys();// Exchange and generate the secret...
const aliceSecret = alice.computeSecret(bobKey);
const bobSecret = bob.computeSecret(aliceKey);// OK
assert.strictEqual(aliceSecret.toString('hex'), bobSecret.toString('hex'));
DNS: Resolve4 Example
My favorite so far. This example code resolves the IPv4 address of archive.org and then does a reverse lookup of the addresses.
const dns = require('dns');dns.resolve4('archive.org', (err, addresses) => {
if (err) throw err; console.log(`addresses: ${JSON.stringify(addresses)}`); addresses.forEach((a) => {
dns.reverse(a, (err, hostnames) => {
if (err) {
throw err;
}
console.log(`reverse for ${a}: ${JSON.stringify(hostnames)}`);
});
});
});
FS: Sync
Read a 600mb file into memory with readSync.
const fs = require('fs');const data = fs.readFileSync('/Users/.../movie.ts');console.log(data);
FS: Async
Max currently uses 12, which was experimental support for top-level await, so I surrounded the function with an async IIFE. Since there are no other tasks, both Sync and Async cases are equivalent.
const fs = require('fs');(async () => {
const data = await fs.promises.readFile('/Users/.../movie.ts'); console.log(data);
})();
A Very Long For-Loop
Here I used a Synth with a sustained note again, which stays on as the event loop stays open waiting for events, until we kill it.
for (let i = 0; i < 1000000; i++) {
console.log(i);
}Next Steps
This has been a fun idea to try out, and helped me better understand a few good tools. A next step might be to actually buy Ableton Live and Max for Live. These scripts, if made well, could have important applications for accessibility.
Developers who can’t take too much information bandwidth visually could take advantage of audio feedback of their runtime. Something like this can be integrated as a debug tool in IDEs, as a language extension, or even hooked at the system level.
As a fun project, another interesting thing to try would be to generate abstract visuals, which can be done in Max as well, with Jitter.
By the way, I also hooked the MIDI output to my Korg digital piano and got some interesting music from my micro-services, I will record it in the future.
Try it yourself!
Linked is the repo of my node fork, with the instructions on the README file.
I thought about bundling the project as a Max for Live Plugin and publish it on https://maxforlive.com. However, this plugin requires compiled binaries. I don’t know if that’s allowed on the platform, but it definitely shouldn’t be, for security reasons.
Do you wish I had tried something else? I’m curious — let me know and I might try it out. Either in the comments or as a GitHub issue.
