The Engineer’s Gift
In Which a Programmer Gives the Greatest Gift of All
This article is exclusive to Pointer — a reading club for developers.
Sign up at Pointer.io for our weekly newsletter.
Dear Recipient,
Happy holidays. Enclosed please find a gift from me to you in celebration of this holiday season.
Because you mean so much to me, I did not want to simply buy you a random thing. Instead, I wanted to find the best item in the world that truly represents you as a person, me as a software engineer, and us as a combination of the two. The problem, then, was to optimize for these three constraints.
So I wrote a program. This program runs in the cloud, across multiple servers distributed world-wide, crawling the web, gathering data, analyzing that data, filtering results, calculating outcomes, and reporting them back to me.
The program took a series of inputs provided by me about your personality and preferences. For example, I know that you prefer blue over red, that you don’t care much about automobiles or weather patterns, and don’t floss regularly. This ended up being 54,212 data items, taking me days to enter and weeks to refine. Every day I think of another few items that have to be added to the inputs (for example, I remembered this morning that you don’t like the sound a cat makes when it coughs up a hairball).
I also entered information about myself. For example, I am somewhat technical by nature, prefer programming to conversation, and would rather wear a hat in the rain than carry an umbrella. I could only come up with 27 individual items here, which saved significant time in this pre-processing step.
I then entered information about the two of us together, such as where we live, the things we enjoy doing together, and our relative heights. Once again, the inputs were many (23,481) and took a significant time to enter.
To speed things up, I created a voice-recognition module for the program so that I could enter items wherever I happened to be, but my voice gave out in my car three weeks ago (which explains why I have only contacted you via text since then).
Finally, I provided monetary constraints, because I lack infinite resources (particularly after the resulting compute and storage cost overruns of my project).
The program first crawled the entire web searching for online merchants (247,363); obviously, if I were to purchase something, it would have to come from a place which sold things. I’ve heard that there are places that sell things that aren’t online, but I decided not to tackle that factor in v1 of my program.
Given this large set of online sellers, the program returned the superset of all items available for sale (4,563,744,809,522,333,626, after de-duping redundant items). This large amount of data required huge amounts of storage across multiple redundant disk arrays.
The algorithms in the program combine all of these inputs and determine the one, single gift that best obeys all of these constraints. This, then, would be the gift that I would give to you this holiday season, perfectly representing everything you are to me.
Unfortunately, the program is still executing. It turns out that analyzing all of the world’s purchasable items to extract a single, perfect result takes time, even with the many clever optimizations that I added. The data alone caused problems that exceeded today’s compute abilities. For example, the inventory of merchants and items created a dataset larger than the capacity of current databases (and required storage capacity far greater than my allocated storage (and savings account), requiring me to implement a neat hack to page the data in and simultaneously run a progressive filter on it). Finally, the massively-parallel computations necessary to run all of the constraints across this huge amount of data are taking much longer than my original estimates. By the latest calculations (from another script I wrote), the program should be able to deliver a result by Christmas of 2071 (factoring in reasonable Moore’s Law advancements in computing in the meantime).
So instead of giving you the perfect item determined by my program, I give to you my program instead. They say it’s the thought that counts, and this one counts very, very high.
Happy Holidays,
- Me
