Ky’s (Somewhat Poignant) Guide to Asynchronicity and Cryptocurrencies — Part 1
(Also inconsistently referred to as Ky’s Guide to Starting Over 1.8.1 or Ky’s Guide to Event Loops in future annotations and subsequent editions)
Time for a New Hobby
The wind blustered outside, throwing fistfuls of icy snowflakes at my windows and carrying off a few strips of siding when I responded by shuttering my shades and turning up the thermostat. Winter was here in all its finger numbing, nose-boogeying majesty, and this time around, I refused to play the game. I had shuttered myself in my flat, a few pallets of tinned stew to keep myself fed, and some dried fruit strips to prevent any bouts of scurvy.
I had also found a new interest that was able to scratch an itch on the hairy midback of my brain that marble collecting was just not able to reach. You see, I had found out about cryptocurrency just as winter was unfurling it bivouac in the front lawn for a nice long visit. It was revelatory; a decentralized set of currencies that I could use to get my money out of the hands of those cunning, suit and necktie wearing spiders at the Gotsham Regional Credit Union. Instead of parking my money in the untrustworthy clutches of some cabal of investment bankers, I could take my physical currency and put it in… the blockchain… no wait, in the… cloud…?
Ok, truth be told, I wasn’t exactly sure how the whole thing worked, but I knew that it was decentralized, and that I could use it to have digital currency. This meant that I could unbank myself and keep my assets in cryptocurrency and another physical commodity that has always and will always command high trade-value, marbles.
Data Based Decisions
Once I had decided to start looking into cryptocurrencies, I realized that prices fluctuate often and sometimes wildly, so it would behoove me to track prices of my target currencies, to know when to trade one for another, or when to sell some marbles to pick up some more crypto. Who knows, if the German Symmetrical Clambroth market dips, I could always sell a bitcoin or two and clean up on the inevitable rebound!
I decided to build an interface for myself; using this interface, I could easily check the current prices of a few selected currencies and use that data to make investment and trading decisions. Data based decisions, that was the name of the game!
Sure, I could just go to some website and check the prices manually, but that’s what Joe Everyman would do; go to one website, click around a few times, get what I needed, then move to another site, repeat. No, I was going to build my own interface to get the pricing data that I wanted with the click of one button.
To do this though, I would need to leverage the power of an API, one that I could use as an interface to real-time cryptocurrency data. Luckily enough, one such API exists. coingecko provides a wide array of APIs that can be used to get all kinds of information about all kinds of cryptocurrencies, and it is free to use!
With the API picked out, I read through the documentation and tried plugging my code in. I quickly realized from the slew of errors that were thrown, that I was doing something wrong. It appeared that I did not take a few things into account. The first and most major was this: an API call to an outside data source needed to function as an asynchronous operation, and I hadn’t accounted for this in my code, having framed everything in the perspective of sequential, near-instantaneous evaluation. The second was that I had underestimated the vengeful nature of those bankers at Gotsham, and needed to spider proof my flat.
Thinking Out of Sequence
Asynchronous code was something I had not accounted for, much to my embarrassment. But to be fair, JavaScript can be kind of complicated with how some operations are resolved. It ultimately comes down to whether the code is synchronous or asynchronous. I realized that I had not been considering the elements of JS that surrounded these kinds of operations, and why that would cause issues with a basic request to an API. It would be important to understand these pieces, as they would ultimately inform what I could do with my code in this circumstance and how I might do it!
To start with, I had to remember that JS is evaluated sequentially; I mentioned this previously, but what did that actually mean? It meant that code in any JavaScript file is read line by line, in sequential order. Consider the following code:
Each line is read in order from top to bottom and each line is executed in that order. Sure, if there is a closure or variable referenced down the line, JS might hop back up to its location to complete execution, but it does not deviate from the order in which it reads the file: top to bottom, one line at a time, in order.
This sequential processing is important when I also considered that JS is a single-threaded language, it can only run one operation at a time, ever. But what did that mean when it came to building out an app that could get pricing data from the chosen API? Well, consider this little conundrum:
If JS runs sequentially, and it can only do one thing at a time, then when it encounters a line of code that might take a few seconds to execute, the entire program comes to a standstill while that is happening. This execution bottle-necking can have huge impacts on the performance of the program, and its usability.
Luckily for us though, there are folks who were savvy enough to realize what a huge problem this might be. After all, how could one expect an avid marble collector to sit for several seconds while their marblemaniacs.net homepage loaded in small chunks as several small API calls were made to various marble collecting databases? It would be an untenable, uninspiring experience.
This is why there are components that function with the JS runtime which allow us to perform some operations synchronously and others asynchronously. Synchronous operations run following the sequential order of execution, while asynchronous operations run by starting execution when they are encountered, then completing their execution after some period of time has elapsed. Typically, this elapsed time is the time it takes to make some kind of request and receive a response, but we can also use a useful method to demonstrate how asynchronous code works without having to worry about all that just yet.
Side Note: The mechanism for asynchronous callbacks for JS is not actually provided by the runtime, it is a sequentially executed runtime after all. Rather, the browser in which JS is being executed will provide the Web APIs to make concurrent operation possible (or C++ APIs for Node).
Timeout
While working on my price checking application and stumbling through issues with mishandling asynchronous code, I thought it would be best to go back and reexamine just how async code was processed. After all, rushing only led to products of poor quality, right?
The method I could use to help visualize async code was `setTimeout`. It is a Web API provided by browsers (refer to the previous note on browser APIs versus JS runtime native features), and I wanted to use the browser as my framework, as I would be using browser-based JS for my app.
`setTimeout` is simple enough to understand; it takes two arguments, a callback and a number. The callback is executed after the number of milliseconds has elapsed.
Check out this example:
We can see here that even though the `setTimeout` invocation and its callback are encountered second in this block of code, the callback is not executed until well after the last line of code was executed and ‘Position: 3 — Executed: Second’ was logged to the screen.
This makes it obvious that the invocation of `setTimeout` is being handled differently; after all, JS is single-threaded and sequential, so if it weren’t being handled differently, the program would bottleneck until the delay was resolved. But it doesn’t! The program execution continues to flow and the next line is executed as the delay for `setTimeout` is being resolved!
That is the beauty of async code; we can let the async function do what it needs to, and once it is ready to resume, it is joined back into the normal flow of execution.
We can see how JS handles multiple async calls that might occur in the same program:
There is more than one async call (to `setTimeout`) and one takes longer to resolve than the other, but there are no bottlenecks and no errors. The async functions are handled and their callbacks executed once their delays have elapsed.
This was all well and good, but it seemed to open the door for another major question in understanding async code. How does the browser create an environment that allows for concurrent execution when JS is a single-threaded runtime?
Stacks, Loops, and Queues
My shade flitted ever so slightly, as if pushed by the gentlest of breaths, signaling that despite my best efforts, winter would find a way to make itself known, even if I had closed myself inside, curled under my warmest jumper. I was eating a bowl of Loops and Queues, (quite popular over here, I assure you) and thinking about just how browsers handle the processing of async functions that are encountered in the JS runtime. Clearly, they could handle code that took extra time to execute without totally bottlenecking all other operations, but what was the mechanism that allowed that to happen, you might ask?
This is where the cooperation of a few separate piece come into play. The JS runtime’s Call Stack and the browser’s Web APIs, Event Loop, and Message Queue all work together to create an environment that allows for some degree of concurrent execution despite the single threaded nature of the JS runtime.
Prepare yourself, as this might seem a bit dense. Don’t fret though, visual accompaniment to follow:
The JS runtime passes stack frames to the Call Stack where they are executed in the order that they are encountered. It operates on a last in, first out basis, so the most recent call goes on the top of the stack and must be executed before any calls further down on the stack are executed. I equate this to my reading list and the pile of old books on my desk; I can easily add another book on my “need to read” list and toss that book on the top of the pile, but that means that I now have to take that book off the top of the pile and read it first before I get to any of the others (or the rest of the pile will fall apart and my list will become an incomprehensible tangle of dusty pages).
We can show how this process works with an extremely simplified model of the Call Stack:
Based on this example we can see that the method invocations that are encountered must be pushed to the stack in the order they are read and executed, then popped from the stack as they are resolved. This fits with what we have already learned about the JS runtime’s handling of function invocations. But, what did this model mean for an async call? If it runs asynchronously, then it can’t possibly sit on the call stack, that would cause a bottleneck.
It doesn’t! An async call is handled by a browser Web API that pulls the callback from the call stack, runs the async operation, then passes the callback over to the Message Queue. But how does the callback ever get executed if it gets pulled from the call stack, and passed over to the Message Queue? That’s where the Event Loop comes into play. The Event Loop is a mechanism that acts as a go-between for the Message Queue and the Call Stack; its job is to monitor the Call Stack to see when the Call Stack is empty. When the Event Loop sees that the Call Stack is empty, it will check the Message Queue for any callbacks that are ready to be evaluated. If there is a callback waiting, the Event Loop will then pass the callback over to the Call Stack to be executed.
I fumbled through my notes and tried to remember how this all worked. An empty tin of stew clattered to the ground as I knocked it over, along with a leaning pile of books. It’s one thing to talk about this complex chain of browser mechanisms and runtimes, but having a solid mental model was an entirely different thing, kind of like saying you’ll eat pickled herring is one thing, but when your neighbor comes back from Oslo with a jarful of the stuff…
Luckily, I did find a powerful tool for visualizing these relationships among my scattered notes. Loupe is a free tool that can visualize exactly what is happening at each stage of an async call. I plugged away some code in Loupe and solidified my understanding of the Call Stack, Web API, Message Queue and Event Loop. And with that… I felt ready to begin addressing some issues in my application.
An Inconclusive Conclusion
A greasy droplet of stew gravy dribbled down my chin and splotched into a nice new stain my pair of well-worn slacks. I didn’t care; at that point I was on the verge of getting my cryptocurrency application all figured out. Now that I knew what asynchronous calls were and how browsers and the JS runtime interacted to make such concurrent operations possible, I felt like I was ready to figure out exactly how to get my application working.
Sliding the finished tin over amid a growing pile of its compatriots, I pulled up a few different websites, cryptoChecker.com, marblemaniacs.net, and a few others; I sorted through them and noted the prices of the cryptocurrencies that interested me, like Bitcoin and Ethereum, and checked the prices of a few highly prized marble varieties. Unfortunately, I had to do this the hard way, the manual way. But not for long. Soon, I would have my CryptoChecker app functional, and I could get all of this done with one click.
However, there was another piece that I needed to work out before I could make that happen…
To be continued, or not… We’ll see.
Until next time, or not… We’ll see.
-Ky
Postscript
The story continues. Find out how XHR is used to get the prices Ky is looking for.
