Not invented here, revisited
When should we reinvent the wheel?
Mnemosyne
Recently, we redesigned the main storage architecture of one of our products.
We did so by writing a distributed bitmap index of our own that was exactly suited to the type of searches and aggregations that users perform frequently. My colleague, Phil Messenger, who served as the lead engineer on the project, gave a technical talk at Devoxx 2018 about how the system was conceived and designed.
While this project was running, and despite my absolute trust in the team’s technical ability to deliver it, I couldn’t help feel that familiar nag in the back on my mind: are we definitely sure that this hasn’t been built elsewhere? We were convinced that it hadn’t, but it is always hard to be one hundred percent sure.
Since it was a new distributed system, development and production rollout was challenging and proceeded at the cost of developing less customer-facing features. This reality was a harder sell to our product team, as our application, at the time, was…
