If it works it works — cowboy code in startups

Being a developer, it’s hard to resist finding a way to do things better, a better process, a better design pattern, better networking..etc but what I often forget is that a startup is often not an organised company.

As a startup, you try and take every shortcut to get things done, without sacrificing any security or efficiency in the end result, but the code itself doesn’t have to be simple. Almost every startup I have joined have their legacy code, things that barely work and need to be rewritten, old outdated technologies that need to replaced by new ones, there is a reason that every startup has that: They were focused on making it work

A couple of months ago I decided it’d be good for my app My Guinea Pigs to implement microservices in our backend, microservices is an architectural pattern that allows every service in your backend to be modular, in other words, every feature is completely isolated which makes it easy to copy, delete, replace and add new services.

That meant that we needed to rewrite our entire backend (a working one at that) and rebuild the database and migrate all of our users and data from MySQL to DynamoDB. I got my team excited and convinced and we went ahead with it.

Techy alert: The next part of this article is a bit technical

First, we had to plan out everything, thinking this is a simple architecture change and the code itself won’t need much of a rewrite other than connecting to a different database and have a different way to get the data. Claiming that moving from a relational database to a NoSQL database is not so simple is an understatement, it was extremely complex that we had to rethink of how our data flow is going to be like and what kind of data do we need at each stage.

Mind you most of our services had more than one DynamoDB database because it caters for more than a single model, i.e. we have a pet profile feature where users add in the details of their pet along with logs of their weight; our pet profile service had one database to contain the profile itself but we didn’t want to clutter that database with the weight data, mainly because one view in the mobile app does not need to see all the weight log at the same time but there’s another view that does, so instead of dumping all the data into one great object we created a second database to contain the weight data, so our service looked a bit like this:

Now there was the problem of making sure that all databases are consistent, with duplication of data (which is sometimes necessary in a microservices architecture) we needed to have one of the databases to act as a source of truth and move the data over to the other databases, hence we needed the databases to be event-driven and look something like this:

Luckily for us, DynamoDB has a feature called datastream which is basically a script that triggers once the database receives a create/edit/delete action. Using that feature, our databases now became event-driven within each service

The techy part is over, back to the fun stuff

I underestimated the amount of work needed for that rewrite, and even though the rewrite and the new architecture is justifiable by being a better architecture it was a process that took us weeks to do and by the end of it, we all agreed that it wasn’t necessary, at least it wasn’t necessary right now.

The new microservices architecture will have its perks, absolutely! it will allow us to build things quicker and reuse more code and it will pay off in the long run but we could have used these few weeks to build badly needed features first, then do this rewrite later on. This is one of the times where I jumped the gun and got my developer side in me excited about getting rid of the old architecture and building a new one without paying too much attention to the business needs.

A good architect would know how to improve the architecture and improve its effectiveness, efficiency and all those nice words that give management the jiggles but a better and wise architect would know when to do so; picking the right time is as important as the change itself and keeping the business goals above having a good architecture, most times the two would align, after all the business can’t operate well without a solid infrastructure but when the business needs don’t align with the architecture’s timeline then biting the bullet and meeting those needs with the bad code/network/architecture/process remains the more important task.