Don’t Defer Load, Offload It!
At Social Tables we have a problem…our users love PDFs! CPU sucking, memory intensive PDFs. These PDFs are sometimes much more complicated than what is nicely handled via “print and select pdf as printer” instructions and so we can’t just port the heavy lifting to their computers. We have to crunch the things ourselves. Did I mention that our users love PDFs? That at almost any time of any day there are requests for these memory sucking things hitting our servers?
First Solution: Lets not really worry about it.
We knew that PDFs were going to cost us a bunch of memory, but scaling vertically seemed OK for us at the time. I mean, how many PDF requests are we really going to get? For about six months this worked. We’d hit full CPU from time to time and we knew things would have to change, but there were other fish to fry and so it stayed. Then we on-boarded all the Hyatt hotels in North America and peaking (eg: CPU/memory usage reaching max) became the norm. Things had to change.
Second Solution: Kue
Panicked, we looked for a job-queue framework with a quick learning curve. We landed on Kue. In retrospect we should have spent more time looking and thought more about the many posts we had read about things such as zeromq, but we were BOGGED DOWN BY PEAKING SERVERS. Lesson: don’t wait to handle problems until you are limping along. It often leads to bad decisions.
Kue worked for us for a while. We had problems over and over again and some of the nicer features of the system crapped out in the first week, but it was working and so we let it keep. That is, we let it keep until we learned we would be ramping up on-boarding and could expect a crazy increase in PDF traffic as a result.
So… do we really want a job-queue?
Knowing that soon our Kue solution would die in a peaking ball of fire that would lead us to rush into another half baked decision if we didn’t do something now, we sat down as a team and starting poring over blog posts we had read about the ways other companies handled the same types of loads. One blog post stood out: Nodeload2: Downloads Reloaded.
Github's repo to tarball service and our PDF service share a need to deliver the result of the jobs to the same request that kicked them off. This is something that typical job queue implementations lack. Designed around managing load, job queues accept “messages” which they then give out to workers as the workers are free to work. We can’t have workers responding to jobs as they are able, we need to respond as the job as it comes in. Our need isn’t limiting load, its offloading load away from our most important services.
With this need well defined, we started looking for frameworks that would easily work for us. Our current routes were all written in node so we looked for Node frameworks that matched our needs and in the end NPM came up wanting! A week later we had pushed “Offload”, our new service to handle this problem into production and today, after a month of tweaks, I am happy to announce we have open sourced Offload under the MIT License.
What Is Offload?
Offload is a small wrapper on top of koa and koa-router. Along with the standard HTTP verbs, Offload adds job, stats, permitGet and permitPost. The full docs can be found on github. We are still actively working on a few features and welcome any you want to add. Please feel free to Fork, commit and PR as often as you like!
One issue that I’m really excited about is a dashboard integration. We have some code in the works for geckoboard and I hope to see that go public in the next couple of weeks.
If you have any questions please feel free to reach out to me on twitter. @mcwhittemore
