Why I over engineer.
or why I shouldn’t…
I’ve noticed there to be at least 2 schools of thought on this subject, and thought it might be an interesting topic to review and see the pros and cons of both sides of the debate.
Because I like to start with the more convicting side of the argument first, I’ll start out by discussing what I think are the biggest draw backs to over engineering.
Draw backs
It will cost X (time, money, talent)
Probably the most obvious point, is that most of the time you are being paid to write the code by someone else. Unless you have been hired under a different agreement, most of the time you have been hired to write code that most directly correlates to creating a valuable product. This often involves writing as little code as possible, with enough test coverage to satisfy the shipping a (hopefully) well defined software requirement specification so that you can move on to the next requirement as efficiently as possible.
Now, every company has different values and beliefs on code quality vs. project constraints (time, money, etc…), but for the most part, it’s safe to say you are getting paid to make a product.
No direct value added
This point is debatable on it’s own, but essentially if you are writing more code than is required to accomplish a task that could have been done in half the time, then all that extra time spent on creating a fantastic abstraction has now been “wasted” on what could have been the next deliverable feature to the product.
Now at this point, I will stop here because it’s safe to say that any further points will also somehow be related to spending of time, money or talent. So let’s move on to the benefits of over-engineering.
Pros
I will start this section by saying, along with each of these beneficial reasons for over-engineering comes a balance, but I’ll get more into that in the summary.
Cleaner Code
If you are able to spend enough overhead time ensuring the design you choose for an implementation provides the right type of architecture and abstraction, it can be way easier to reason about a code base.
For me personally, it has some correlation to having a clean working space / desk area, in the sense that if you respect yourself you should also respect the areas you live and work in (including a code base). This can also provide an easier barrier to entry for new team members when on boarding into your system.
Developer Experience
One of the easiest ways to gain true understanding of something is to dive in and experience it.
This applies to code design & implementation in that you are forced to do the research and figure out how to do something (or how to do it better) when you take on improving or designing a new system. This is what my career has been built on, being given the opportunity to use many different software design paradigms and best practices in real code. Without the experiences I’ve had, I may not have truly understood why it’s better to use libraries for some things such as a web server. One of my earlier node projects, I made the mistake of writing an API server from scratch in node using a the tcp module and manually parsing http messages. Yes, it was a really dumb idea to take on, but now I have a true appreciation for modules such as express.
With the right mindset, you can take what you did and look back on it critically to say, what worked with this, what didn’t work and what am I going to do better next time.
Re-usability & Scalability
I leave this one for last because it’s the one that’s bit me in the butt the most in my experience.
This reasoning behind why over-engineering can be useful is that in the future, you can implement another like-designed feature with minimal effort due to the abstractions / modules / etc… you created earlier. The reason also goes, we may have huge surges in traffic or growth of customer base in the near future that will cause a huge panic if we don’t optimize this ahead of time.
This is the reason that in my opinion requires the most thought and care before taking it and running at your code editor. The reason being, this has a very valid reason, that could likely (or unlikely) happen in the future, but that’s the thing. It’s the future… meaning you don’t really know. My qualifications for invoking this reason for creating something is the following:
- Will it take a relatively small amount of time to implement?
- Are there enough people available or is there enough time that couldn’t be spent creating actual value?
- Is the weighed risk / probability high enough to need this implemented?
If any of those are true, then it’s probably fair to say move forward with it given you have the proper go ahead to do it as well.
Summary
All things said, I still find many valuable reasons to spend time designing a thoughtful and well structured system and will likely lean towards that bias given the right circumstances. However, in certain situations or companies, there can be much more value derived by getting something created and shipped as quickly as possible (say for a startup MVP). The real answer is up to the specific situation at hand and some genuine critical thinking to decide what the best path forward is.
Now that I’ve dumped my thoughts on this topic… I’d love to hear anyone else’s thoughts / questions or disagreements with any of the points mentioned or any thoughts on this topic in general, so write them in the comments below!
