Software engineering — are we entering a mediocre age?
Linux is mainly created by one man, C is mainly invented by one man, etc. Software industry is good at creating hero, some true legend. It is always excited to hear stories that a few talents work together and create a piece of software that empowers millions of users or even changes the world. However, I am afraid we are not going to see such cases in future.
The reason one man can change the entire industry is because he works on some fundamental capabilities. Such fundamental capabilities usually enjoy some “pure mathematical beauty” — elegant, precise, concise. There are no tedious details. In order to come up with such fundamental capabilities, it requires a man to have very deep insights into the real world problem; in order to implement such fundamental capabilities in a piece of software that lasts long, it requires a man to have extremely good skills. A legend is admired by doing both, and very very few people can do this.
The need for such fundamental capabilities is decreasing. Software is busy “eating” the world now. We are greatly benefiting from the fundamental capabilities those legends created. In order to “eat” the world, we need to replicate tons of functionalities in the real world in software. That is a lot of work, but not necessary a lot of hard work. We need lots of software engineers. The quality of people always has norm distribution, no matter what industry it is. So gradually, the challenge becomes: how can we develop reliable and performant softwares even if we only have average or even below average software engineers. The solution is creating platform/framework. Such platforms or frameworks are usually done by very talented people (but I don’t want to call them legend; just to distinguish them from the true legend). Platform or framework usually restrict the ways that engineers can do things, therefore, it is less likely to make mistakes. Platform or framework usually hides the most complicated or most creative parts, so that engineers can focus on the so-called “business logic”.
For example, AWS. 20 years ago, if you could set up a website, you are considered as a hacker; 10 years ago, we need very talented engineers to set up and manage your database and network. But with AWS, all these pains are absorbed and solved by Amazon. You don’t have to understand how to build a database, you don’t have to know how database works underneath anymore. All the complexities are hidden behind AWS’s APIs. This trend is only accelerating. More and more layers are being built (ie. software as a service). As a result, it is never easier to create a piece of software today, more and more software companies are blooming thanks to these frameworks or platforms. All these companies are focusing on replicating real world, all they need are engineers who can do “business logic” really well.
If a young generation of engineers is only doing “business logic”, they lose the motivation and opportunities to train themselves to be legends. This seems inevitable. The so-called “full stack” engineer is a sign because no one can be true experts in “full stack”. Human knowledge is not always accumulating. It is absolutely possible that knowledge can get lost if no one is practicing it.
After we enter such a mediocre age, creativity is not the most important thing anymore (I am referring to technical stuff, not product definition), all we need is project management and solid execution. We can still deliver valuable software, it is just not as fun and exciting as before. We will miss our legends.