Do our projects need to be so complex?
I drew inspiration to write this article from the presentation by Fabien Potencier entitled “Boring is the new hype” during the conference SymfonyLive Online Polish Edition 2021.
Fabien showed the photo of his kitchen which looks like the kitchen in a big restaurant. Why has he spent so much money to possess such a big kitchen? There are a few reasons for that: he likes cooking very much, thanks to the kitchen and its equipment he is able to prepare any meal he comes up with. It is much easier thanks to the employed staff and a variety of appliances.
Unfortunately, there are some disadvantages of this solution. He doesn’t know how to use all of the utensils in the kitchen on his own. He also probably doesn’t use some of them in an efficient way or just uses them incorrectly. He also has to take care of the staff, sometimes to hire or fire someone. The kitchen needs to be constantly supplied with fresh products. And the biggest problem is that his children eat only pasta and pizza.
Of course, the fact that this is Fabien’s kitchen is not true. He made it up to force us to mull over technology and our projects.
Such businesses as Spotify, Netflix, Facebook, or Google encounter much greater challenges (but not problems) on a daily basis than most of us do. But our projects are very often much smaller, with less traffic, and fewer users whose accessibility doesn’t have to reach almost 99.999%. We, the “small” ones, very often rely on the technologies and tools developed by the aforementioned companies. Those businesses were forced to create some new tools, non-existent at that time, to solve their problems.
Most often the tool is adapted to the requirements of the big companies, so it is not necessary for our “small” projects.
Choosing the technology and the tools is always a question of choice. The decision that has its context. Are we developing another CRUD for the category view? Or maybe we are briskly preparing MVP for the client who wants to see how something is going to work during production? Of course, later on, this MVP becomes a manufacturing feature, with some further objections, like why is this functionality ineffective or why doesn’t it have nice colors?
The decision on how we are going to supply some solution is always accompanied by some context. What is our budget? How much time can we devote to it? Is it an internal project only for our colleagues or an external one, so we need to take greater care of its security? We need to assess the experience of our team, the skills that its members possess as well as the ones they lack, the amount of time required to learn this new technology, and the approximate amount of errors we are going to make while using something new. Is it possible that while choosing a specific technology we will need DevOps’ support? Is it possible within our company? Will these people be able to devote the time we need to us?
Our task as developers consists of developing the product, adding new functionalities and maintaining it, and not learning some new fancy technologies.
And this may lead to the conclusion that for our current project, the best solution may be some boring and well-known technology/tool. Something that has already been available on the market for some time, from the manufacturing point of view already used by many people. Something tested by many more developers. Something reliable that has a rather secure future. Something that is (what a surprise) documented, and what is more, it’s up to date! Something possessing big community and thus it’s easy to find the solutions to our problems. You can find everything on Stack Overflow, can’t you? ;)
An example of such a solution can be constituted by a good old SQL database. At this point, it doesn’t matter whether it will be MySQL, PostgreSQL, or something else. Such database is usually included in our project, one way or another, for storing basic data. It can be easily used for storing messages from the queue system instead of Rabbit. It can be used for storing cache instead of constructing Rabbit. Will it be effective? Probably not, but it’s usually enough for the beginning and it’s going to meet our requirements. What do we gain from this? Probably, in most cases — time. We don’t have to develop our local developer environments with more containers for docker with Rabbit and Redis. DevOps, administrators, or other people don’t have to prepare new helm charts for our trendy environment built on Kubernetes. We won’t have to warn everybody that during the first implementation on production for sure something’s going to crash and there’ll be downtime.
So if we gather all the errors that happen to your application in one place, will you choose Sentry? Or maybe Rollbar? Or maybe NewRelic to have even more opportunities to monitor your application? Or maybe you’re going to build your own solution, especially for this purpose? I really doubt it, in such case, we would rather choose some ready-made solutions already present on the market and we are able to select those ones that are the most suitable for us or those ones that we can afford after a few hours of explaining it to the business that, all in all, it’s nice to have something like this. Such solutions often set us free from such obligations as their installation, monitoring whether they work properly, updating them to the latest versions, and maintenance of the server on which the solution operates. We most often perform simple integration and it’s ready to use. Similarly, the question is whether we are going to build a new payment gateway or mailing service responsible for sending thousands of e-mails per second.
We’ve always been quarreling about which programming language is the best. PHP? Python? Ruby? Java? We know for sure that it’s not JavaScript ;) The situation is similar with databases. MySQL or PostgresSQL? AWS or Azure? RabbitMQ or Kaffka? These quarrels are usually of little significance as both solutions are capable of providing the client with what they expect at the very beginning, even before they actually know how the project is going to develop. As developers, we can be sure of only one thing — of changes. There are always changes, but the client can surprise us with the direction in which these changes go.
Do we really need trendy Kubernetes from the very beginning? Before millions of users begin to use our application (because our idea for this business is so amazing, isn’t it?), weeks and months will pass before we build the product that we’ll be able to show to our potential clients. Maybe it’ll be the best idea to use the simplest server. K8s has many advantages, but it isn’t so easy, simple, and fast to use. One needs to gain experience in order to use it in an appropriate way. In the PHP world, we could recommend the simple old LAMP set plus, of course, Fabian’s favorite framework — Symfony. It is old and boring, but thanks to this fact, very well tested and documented and its community is huge, so we’re able to get assistance very fast in case there are some challenges (remember, never problems).
Please find below a real-life example of how you can make your daily work harder.
The project, a monolith, one repository. Functional tests? A separate repository, other git flows adopted, other branch names. Always thinking from which branch to perform tests on the dev environment? And from which stage? And has QA already updated the tests? Or not yet because they were doing the regression for 2 weeks on production after the last issue? Is this error an error? Probably not, move it forward. Or perhaps we could put it in one repo where on a given branch both the developer and QA could perform the changes. In that case, it can be easily verified whether the code works properly and if the tests also work as we’d like them to.
The project is divided into 4 microservices (OK, macroservices), plus 3 or 4 separate repositories used by each of the macroservices. What does it look like for the developer on a daily basis? Commits to several repositories, replacing package versions and only then testing. Deploy? It’s always been easier to push it all at once to production — let’s ignore that something can go wrong. Finally merging the products, reducing the number of repositories, and easier work for the developers on a daily basis.
Additional service for communication with Cassandra and Amazon S3 and for storing the information about images and uploading by users themselves. I don’t know what the author had in mind here, but, for sure, wanted this solution to be prepared for heavy traffic, after a few years of operations of this functionality in the database (and on AWS), there were slightly over 300 photos. What did the author have in mind? I’m still wondering.
While making the decision on selecting a new tool, let’s sometimes think about whether we won’t hurt ourselves with it in the future. We’ll lose a lot of time and, thus, money. And on the other hand, isn’t it a good idea to change something in our project, resign from something in order to make our daily work more pleasant?
