Javascript for machine learning? Get a real language.
A story of loss and the love of innovation
A podcast
Recently I was asked by Gant Laborde to be on an up and coming podcast, and I, after not knowing what to provide for discussion points, began reflecting on some of the open source work I’d been doing… I offered as a topic of discussion for the podcast: “What if Machine Learning was simple enough for children? A journey in making it happen.”, and it became clear as I began typing, this story needed to be shared.
The only reason I can fathom Gant would have asked me as a guest to the podcast is from the work I’ve put in to a couple GPGPU and machine learning software libraries (the podcast is about Machine Learning), and perhaps my own stubborn will to refuse the idea that current machine learning libraries, specifically neural networks, are easy to use and easy to understand. They simply aren’t, and it is total garbage if someone tells you that, or at least… I can say it was.
Let me start out with a clarification to the topic I suggested for the podcast: I don’t mean machine learning libraries on top of machine learning libraries, on top of machine learning libraries, that make them easier to use, so that a child could use them. What I mean is this: what if they were implemented in a way that a child (we’ll say 10 years old+, with a basic understanding of multiplication, division, subtraction, addition, and light programming) could understand the nuts and bolts behind the API that we build it with?
- On the outset:, the benefit is a child, or an adult, or a professional, can all understand them. Sweet! That is great.
- But going deeper: What if because of them, good gracious farfegnugen, someone {dramatic pause here, then said slowly} innovates
The State
The current state of how most serious machine learning libraries are being built is generally so very very close to the metal. And with good reason… for performance. Because of this, we get an API that someone somewhere else thought of, and are commanded to use it and in a very specific way. The “metal” here is like c++, glsl, opengl, vulkan, metal, you get the idea.
On a business logic level we just have (been told) to accept these APIs. By the time these libraries are exposed to a higher level language, such as python, or (yea, I’m going to say it!) javascript (or typescript), there are so many levels of abstraction that, in order to accomplish the suggested podcast topic it becomes so very difficult for someone to innovate, or even just to have a basic understanding of how it all works, because of the extreme learning curve.
But, it doesn’t have to be that way…
The Dare
What if the higher level language such as javascript composed the lower level language such as c++, directly, AND executed it where it was most efficient? This is the big idea, the revolution that I’ve been working on for over 3 years, with tons of success. But it isn’t my idea, I simply saw a glimmer of this vision, and decided to push it forward.
Who would ever dare come up with such an idea?
It turns out, a few friends in Singapore, and they ended up building it…
In 24 hours…
And they won second place.
I’ll come back to that story.
The Search
Somewhere around four years ago I began a search. I know javascript, and I wanted a machine learning library with serious performance, written in javascript. Easy right? My greedy selfish desire was so that I could understand it, and use it. Graphs on Wikipedia were fairly straightforward, and I had become familiar with them over the years:
I searched the web for hours, then days, then weeks (not continuously), and found a few libraries that had fantastic API’s, BUT, they were essentially considered “toys”. Looking further into why, I found that they simply cannot handle the performance required in serious, strenuous, BIG DATA applications.
At one point I started talking with a community of developers who used the most common tool sets, in order to reverse engineer them for javascript. I won’t mention which community. But as soon as it was found out that I was endeavoring to improve javascript, I was told the title of this story:
“Get a real language.”
The Loss
That hurt, as I remember the days of walking away from C# in favor of javascript, because there was no humanly noticeable build time. How you could choose to make things strongly typed, even though the language is weakly typed. The simplicity, the elegance, the innovation. How it had become my primary means of programming for over 10 years. How the open source community around javascript, though imperfect, is generally very helpful.
Javascript is fantastically fast for what it is, a CPU based, high level, language. So when you process graphics (pictures, video, and we’ll throw in here “large amounts of data”) using it, the processing unit on which it generally runs is severely taxed and performance is one of the first things to go. There are ways to alleviate this, such as multi-threading, but the amount of overhead to take advantage of multiple threads, though faster (technically it isn’t faster, they process at the same speed, but the workload is split and executed in tandem, which gives the illusion that it is faster), doesn’t change the architectural limitation. The CPU is designed to do one. thing. at. a. time.
There is another architecture that is designed for massive concurrency. The graphics processor… The “GPU”. Why is it better at concurrency? Lets let Mythbusters explain that:
It was around this depressing time, I stumbled back (I had seen it a few times) to Brain.js. When scrolling through the source code this time around, I (like many others) felt that it had a world class API, was ridiculously simple to understand, and was wildly popular (for javascript)… And then I read that it was abandoned.
Shocked, I went to the issues page to see if there was anything being done about it. There were just issues, and a question as to why it was abandoned.
Rather than go into that story as to why, we’ll stay focused on this one… I’m not sure if it was the shock or the depression, but I began to have an idea that just because the library said it was abandoned, and didn’t yet have GPU capabilities… that it could, and I could be the one to do it, or at least help ;). I could see to it, to continue the work of the library, to continue its story…
But how would Brain.js ever run on the GPU, and be composed from javascript? This was the second search, the search that brought me to a wild idea about innovation. The “not my idea” idea.
The reply
If I could build (or improve) an environment that:
- computed the low level source code needed to run on the GPU
- allowed me to compose in pure javascript (or typescript)
- that had serious GPU performance
Then I’d be doing something brand new and widely needed. Then we could innovate in javascript for these types of problems because it could be widely understood. It is important to mention that this is not just on Machine Learning applications, but rather a new paradigm of programming, one that enables someone with hardly any experience, to write fantastically sophisticated software that takes full advantage of hardware, an example being the device you are reading this with.
There was a problem, the 24 hour hack-a-thon project, GPU.js, was really just a proof of concept, its entire feature set was not well tested or documented, and though its creators did a good job with it, it was still… buggy. It was in a very similar state to Brain.js, and it didn’t even run on Node.js. So, to me, it was a 2 for 1 deal. Continue Brian.js AND GPU.js, make it run on Node.js, and only then, will we have seriously fast javascript for machine learning.
Armed with this idea, I could see myself standing up to that negative response “Get a real language”, use it as fuel, as motivation, and learn a lot in between. To which I could finally reply, lightheartedly:
“Get a real language? I did. And its Javascript.”
But how is this approach different?
One could ask the above question, and in fact I did to myself (and still do) before finally getting started. It is easy to not see why the strategy that Brain.js would take would be so different so I’ll quickly explain and summarize because it is important.
We’ll pick a library that didn’t exist then, but does now: Tensorflow.js, a finely written machine learning library that mimics the somewhat pedantic API from Tensorflow. While Tensorflow.js has some javascript, it also contains separately written and maintained WebGL (and soon to be WebGPU) counterparts AND node bindings AND a c++ counterpart. This is important: Those environments and bindings come with technical debt. So for every one of anything, there are at least… three (or more, four, five?) parts that must be maintained.
The downside to this is that it can lead to different environmental behaviors, yes, but let’s go deeper. Imagine I’ve reached the limit of what Tensorflow.js can do, and I have a new mathematical intuition, or layer type, or activation that I want to try, right? Well, I cannot write it once, and forget it. I have to write it once (c++), and then expose it as a Node.js binding, and then write it again (javascript), and again (WebGL), (and eventually again, WebGPU)… That is the opposite of simple.
Imagine writing a novel in the same way, or painting, or coloring, or… you get the idea. I’m not trying to attack Tensorflow.js, I’m simply saying that approach, while Tensorflow.js is doing a great job at making javascript/typescript have a means of machine learning, writing so specific, with all that potential technical debt to each environment is not innovative. I feel very strongly about that. That is/was the “status quo”. Thus it can be said that a very large part of that machine learning library itself, although well written and backed by a company that was not evil, is NOT entirely javascript/typescript.
Now the hypothetical approach that I envisioned GPU.js and Brain.js to take: While GPU.js (a GPGPU library) would need to be written to take advantage of more than one environment (or backend) like: WebGL1, WebGL2, HeadlessGL, and Javascript, each of these are targets that we would compile the input javascript source to. This is VERY important. The internals then of GPU.js would be very general. This would be deliberate. Because of this we could test general behaviors more granularly. Like an if statement or a for loop, or precision.
Brain.js on the other hand (A machine learning library), is written purely in javascript. Yes, the most ubiquitous programming language on the planet. One language, one api, simple, and straightforward. That would be the bar. The requirement to continue it. Javascript would be used as the means of composition, to generate the source for the other backends, and more backends like: WebGPU, Vulkan, and ArrayFire.js could be added at any time. Brain.js would only need to install the newer version of GPU.js as well as any dependencies to gain that new environment.
For the love of innovation
It was an undertaking that would end up taking years, but was and has been worth it.
Why is it worth it? When one innovates, the innovation causes an explosion, an explosion of innovation.
Here is an example:
Harnessing electricity is another example.
Machine Learning is said to be the “next electricity”.
GPU programming is (or has been) notoriously difficult (arguably) to work with in part because of:
- poorly written drivers
- poorly written implementations
- being poorly understood
- very rapidly improving APIs to resolve these previous shortcomings
But if we could write in a single, widely understood, very expressive, very straightforward language, such as meager javascript that compiles down to the lower level languages, then we could have a moving target of a compiler. In other words, we could compile in real time to the best language AND environment of the device that we are on.
This isn’t really a new idea, some examples are c++, python, typescript, and even javascript. These (pretty much all) languages are either interpreted or compile to something lower level, and by relation share this idea, but not in the radical way of such a high level language like Javascript itself composing such a low level language like c++ (or a c++ subset, glsl) on such a different processing unit like the GPU.
I got busy
After hundreds of man hours, over three years of work — around being a husband, a dad, and my day job — GPU.js v2 -“Cosmic Jellyfish” was released last month at the time of writing this story. GPU.js lets you write with very simple javascript that is transpiled to the GPU. It can execute in the browser (Linux, Window, OSX, iOS, and Android) via WebGL1 & WebGL2, on the server (Node.js) via HeadlessGL, and even in native applications (Expo) via @gpujs/expo-gl.
Fast forward as well with Brian.js, the community eventually stepped in to help improve it, to continue its legacy, and I was honored to eventually became the lead developer. From then till now I’ve been working with many others who share a similar desire. We’ve added new neural networks types such as recurrent & time step, improved the tooling, like cross validation, have upgraded the existing API so you have the options to using the GPU in some places, via GPU.js and are in the final stages of finishing convolution, network composition, and recurrent GPU implementations for V2 of Brain.js. We even had an alpha release last month.
However, there has been some pretty substantial delays along the way starting around December of last year, mainly because of boring things like:
- in GPU.js a massive rewrite to account for end to end type inference (yes, strong types in javascript! NOT typescript) — because of Node.js bugs
- writing the unit tests (now over 20,000+ assertions) and fixing bugs — needed to prove the projects and the ideas behind them
- adding the ability for GPU kernels to self-recompile, when mixing pipeline mode — for array index lookup efficiency and stability
- adding Node.js support via HeadlessGL, improving it and being added there as a contributor
- A very long list of other things used to tap into and take more fuller advantage of the GPU
Looking back
I can say that I’ve learned more, been pushed harder, and have overcome more obstacles in the past three years, and more specifically in the last one year, than I have ever. By. Far. And though there have been many people who’ve helped along the way, a large portion of getting this far are thanks to two people:
- Gante Laborde: thank you for encouraging me and being so lighthearted & professional about technology and machine learning, and inviting me for being on a podcast
- The person who told me “get a real language”, because now I can say:
“I did, and it is Javascript.”
Moving forward
Though things seem to get slow at times, we’ve not stopped. We’re getting ready to roll out a new website design for GPU.js, and we enlisted the help of Jay Weeks, Michelle Bowser, and Harsh Khandeparkar (thanks guys!) and we just recently rolled out another website redesign for Brain.js, thanks Muhammad Ubaid Raza, and I can finally get Brain.js v2 released, rather than just keep talking about it!
However, work continues, and we’re still carefully adding new features when needed. But, before rushing to implement anything new, we ask ourselves a question: “Is the API simple enough for a child?”… If it’s too complex, we simplify and ask again. Then, every once in a while we stop and think: “We’re doing something that is far different than has been done, and it’s taking off and helping people… innovate”.
Now that, is a real language.
Closing remarks
Negative energy isn’t a bad thing, it can help us to focus on difficult tasks, or give us energy to overcome great obstacles. I sincerely hope no negative energy comes from this story, only positive. There will be people who tell you “you cannot do that!”, to which I’ve replied many times “I’ve done it already.” But if the task is worth doing, and has not yet been done, prove it to yourself, and DON’T GIVE UP!
In software engineering the “real language” shouldn’t be just one or two of them, it should be all of them. But we know the reality, that one may have better tooling over another, that is simply the nature of improving software. If, however, you find yourself questioning the status quo, see a better, simpler, way of doing things, and are met with resistance, you may be onto something.
Question your idea, prove it, test it, plan it. If it holds up celebrate that, and then, of course, get busy… innovate.
