Software development: what has changed?

379km, about 1 hour “gate-to-gate” (boarding) for 41 minutes of flight.

This is the shortest air route operated by A380, inaugurated in summer 2019 by Emirates between Dubai and Doha.

During this flight, about 14 tons of kerosene are burned in the atmosphere…

Do you find this absurd? Yet hundreds of thousands of companies behave the same way with their IT, most of the time without even realizing it.

Moore’s Law and euphoria

Let’s go back a few years, to the early 90’s, the “micro” is booming in companies.

The “Cloud” does not exist yet and the massive computerization, the beginning of the “digital revolution”, passes then by the equipment.

All large companies want their Datacenter, or at least their server room.
Manufacturers such as Sun Microsystem, HP, Silicon Graphics, IBM, and in France Bull share the Unix server market with their proprietary hardware and architectures, while Microsoft convinces more and more small and medium-sized companies to join their ecosystem on Intel servers, which are much cheaper.

Linux doesn’t exist yet, nor does Google.

The pendulum of enterprise IT is now clearly on the side of CAPEX: investment and amortization.

We buy big servers that we amortize over 3, 5 or even 10 years, whether they are used at 5, 50 or 100% of their capacity.

The software industry is just entering its maturity phase, SaaS and PaaS obviously do not exist yet, and the dominant strategy oscillates between acquisition from software vendors (for an “on premise” installation) and internal developments.

Moore’s law, which foresees a doubling of the number of transistors in CPUs at a constant price every 24 months, is in full swing. Intel released its Pentium in 1993 and will not contradict it for almost 10 years.

In this context of hyperinflation of hardware performance, software performance management is less of an issue than the productivity of developers, who are already a rare and expensive resource.

Language theory research and a number of individual initiatives gave birth to new languages, aimed precisely at developer productivity well before pure performance (it is enough to wait two years anyway for programs to run twice as fast without doing anything except renewing the hardware).

Perl was born in 1987, Python in 1991, Ruby in 1995, Java in 1996 plus a number of other “development environments” such as WinDev (and its teaser ads) in 1993 or Delphi in 1995.

The race to computerization is accelerating, it is necessary to develop quickly, even if it means doing it badly.

The race for productivity

In 1994, the Standish Group published its first “Standish Chaos Report” which analyzed the success rates and the nature of failures in enterprise IT projects.

The results are clear: almost 90% of the projects are failures: either the initial budget was largely exceeded (almost half of them doubled it), or the software had to be released by sacrificing part of the functionalities, or the project was cancelled and buried, without mentioning of course the repeated delays in deliveries.

No other industry would accept such a quality disaster.

In an attempt to fix the situation, new project management methods were invented: the V-cycle, Merise, AGL, and more and more hardware standardization (Intel and Microsoft took advantage of this) and software standardization (ISO model, ASN.1, XML, X500, etc.).

In spite of this gloomy picture, hardware performance continues to grow, almost linearly, with no sign of a slowdown on the horizon.

Hardware sales are doing well, since tomorrow’s machines are needed to run programs developed today.

Language designers are therefore encouraged to push the productivity lever even further, at the expense of performance, and perhaps a little bit of quality as well.

The object model was increasingly successful, interpreted languages were becoming more and more attractive, and PHP, which made its first apparition in 1994, democratized Web development at the time.

More and more abstractions, more and more indirection, more and more layers, whatever the “cost” is, we know that it will be absorbed by the next generation of CPU.

But that was before. Before the drama.

Winter is coming

In 2004, an event occurred that CPU designers/founders had been dreading for a long time.

If the number of transistors continues to double every 24 months as Moore’s law predicts, this doubling is no longer directly translated into an increase of the gross power available, or should I say, easily available.

Indeed, the uninterrupted growth of CPU power was previously expressed by the ability of the CPUs to execute a stream of operations ever faster.

When we talk about “free” performance increase, it does not imply that the new machines were offered to the customers, but that they did not have to do anything special in their programs to take advantage of the improvement (not even a recompilation).

In 2004, the founders reached a thermal limit resulting from the growth in the number of transistors on the one hand and the increase in the operating frequency of processors on the other.

The frequencies have been stagnating since then.

Nevertheless, the improvement in the thinness of the processors’ engraving allowed Moore to be satisfied. More and more transistors were put on the same silicon surface and in the same thermal envelope, but they had to be used differently.

The so-called “multi-core” processors appeared (or rather became widespread).

Before 2004, the objective of founders was to process instructions faster and faster, but after this pivotal year, the battle shifted to the field of parallelization: their ability to process more and more instructions simultaneously.

The graph above clearly shows the concordance of the stagnation of the frequency with the beginning of the takeoff of the number of logic cores. The number of transistors, on the other hand, continues to increase quietly (a dual-core CPU implies about twice as many transistors as a single-core CPU — although it is not that simple in reality).

For the first time in more than three decades, upgrading one’s infrastructure did not allow to increase “for free” the performance of programs…. which were mostly designed to work in a linear way (not parallelized) and therefore unable to exploit the additional cores and the additional of transistors.

The hangover

No more free performance increases, no more systematic absorption of new abstractions, indirections, overlay.

This time, the developer has three alternatives if he wants to improve the performance of his products:

• Revert to lower level languages and optimize his programs: but productivity may take a big hit.
• Use parallel programming: but this is much more difficult, a source of multiple errors that are difficult to predict and reproduce (race conditions, dead-lock, hazardous management of mutex, etc.), it requires a much higher level of skill and is too rare on the market to be generalized.
• Cut large programs into several smaller modules (but still monothreaded) that can run in parallel and thus use a little of the available power.

It is essentially the third solution, the least expensive and the least risky, that will be favored.

New paradigms then appeared: multi-tier architectures, then service-oriented architectures, followed by microservices architectures, which are still in use today.

Thus, a large monolithic program designed in the 90’s / 2000’s can be broken down into small modules (let’s say twenty or so) that will work together, each consuming the equivalent of a CPU core, to deliver the same service.

Obviously deploying, monitoring and operating 20 programs costs more than one, so we had to find a way to automate these tasks to avoid economic disaster: say hello to Docker and Kubernetes.

Not all is lost for everyone…

On the one hand, we have new CPU architectures that companies are struggling to use optimally, and on the other hand, automated deployment and operating systems that are becoming more and more efficient.

Wouldn’t it be a good idea to share a multi-core infrastructure between several companies?

Thus was born the Cloud.

The 2010s saw the emergence of new types of offers, a new revolution that is not so much technological as economic.

Instead of investing in hardware that will be amortized over several years despite a more than partial use, the company can now rent its use for a fraction of its price since several companies will share (in principle unconsciously) the same hardware infrastructure.

From CAPEX we switch to OPEX: and this changes everything for the developer.

The company’s costs will now be directly proportional to the performance of its programs (which many companies have not yet realized).

The better the performance, the less resources they consume (CPU time, RAM, storage, network), the lower the bill from the Cloud provider, which is based on the “Pay-as-you-use” principle.

We are in 2021, and we have to admit :

• That a large part of companies has not yet fully realized this.
• That those who have become aware of it are discovering with fright the extent of the damage caused by 25 years of priority to productivity.

Technical debt

Infrastructure models have evolved much faster than development practices.

Job offers in the developer market speak for themselves: the most popular languages are those of the 90s, designed with developer productivity in mind, sacrificing performance (or efficient use of resources).

For example:

• JavaScript: an interpreted language whose catastrophic performance is matched only by its level of reliability and security
• Python: another interpreted language, adulated by Datascientists and other RAD (Rapid Application Development) enthusiasts. It has at least the merit compared to JavaScript to introduce the notion of typed programming (although it remains dynamic) decreasing a little the spectrum of potential errors.
• Java: launched in 1996, a time when the diversity of CPU architectures was still important, with a seductive promise: “Write once, run anywhere”. It introduces a new concept, it is a compiled language, but for a non-existent CPU. The “bytecode” produced by the latter is indeed executed on a virtual machine (the JVM) which will convert it on the fly into code executable by the underlying physical CPU.

Let’s take a look at the latter, which still represents the majority of business application code in companies.

The initial promise was seductive, in the 2000s, the decade that saw its popularity explode, the server park in companies was still very diverse:

• Unix servers (where each manufacturer had its own OS and proprietary CPU architecture),
• “Wintel” servers (Windows + Intel),
• and still a lot of mainframes for the biggest ones.

The application development was therefore divided into as many silos as there were hardware targets, with dedicated tools for each of them (IDE, compilers, build tool, debugger, libraries, framework …) and dedicated human skills.

This was a real obstacle to the pooling of developments and the rationalization of teams.

The promise of a technology that allows code to be written only once, regardless of the target platform, was a miracle: THE solution for many companies.

Java also popularized a concept that had previously been relatively confidential: object-oriented programming.

The ability to reuse code more easily, whether written by oneself or by others: more productivity gains in perspective.

It has been a dazzling success, it goes from the 26th place of the TIOBE ranking in 1996 to the 3rd in 2001 and obtains the podium of the most popular languages in 2005 (a place it will only lose in 2012).

Its adoption is undeniable and it starts to be taught in engineering schools.
But as in computer science, magic does not exist, its two main qualities (universality and productivity) are once again paid for in the field of resource consumption.

Java suffers from two big problems:

• A gargantuan memory consumption: 20 to 100 times higher than an equivalent program written in C for example.
• A very slow startup time, because the JVM has to analyze the bytecode and its real use at runtime to optimize its transcription into native code.

Greed and finite resources

Java is greedy, very, very greedy, especially when it comes to the RAM of the machines on which it runs.

This bulimia is explained by three properties of the language:

• Its mode of operation: compilation “just in time”: at runtime, which makes massive use of introspection, greedy in resources.
• Its memory allocation management mode: all Java objects are allocated in the heap, the least efficient, and in Java, everything is an object.
• Its object model: while it facilitates code reuse, it also encourages the uncontrolled loading of unnecessary code.

There are two types of resources on a computer: finite resources and infinite resources.

• Memory and storage are finite resources: the machine has a fixed quantity (e.g. 64GB of RAM and 2TB of disk), it does not increase with time (except by physically adding more).
• The CPU is an infinite resource: The number of cycles, and thus, of operations it performs, increases with time which is a priori infinite (although not all physicists are yet certain of this hypothesis). Its speed is constant over time (it does not accelerate as it gets older, but it does not slow down either), but the more time passes, the more the sum of the operations performed increases.

Cloud providers have obviously built their pricing model by essentially charging for the use of finite resources, RAM and/or storage.

A software written in Java, consuming a priori much more RAM than its equivalent written in another language will consequently be much more expensive to run.

Incompatibility with the microservice model

Java was conceived and popularized at a time when the monolith was king, the big software running on a big server that belonged to the company and that only it used.

Its startup time was not important as it was started once and then ran day and night without stopping, whether there were users to consume it or not.

As long as the company was financially amortizing the server whether it was used or not, the fact that it was loaded to 5 or 100% of its capacity was not an issue. The fact that its available resources were being wasted was not important.

Many CIOs in the 90s and 2000s even measured their success and importance by the number of machines lined up in their server rooms. Environmental concerns were unfortunately not yet relevant.

The advent of microservices architectures and orchestrators, for the reasons explained above, has changed the game.

Since microservices are still mostly unable to exploit the multithreaded/multicore capabilities of modern CPUs, the idea was to increase the processing capacity of the system by multiplying the number of instances of each microservice to which a front-end load-balancer usually distributes the work.

Modern orchestrators can easily detect the processing capacity limit of an instance, so they trigger the launch of a new instance of the service and then balance the traffic between them.

When the traffic decreases, the orchestrator will “destroy” the useless instances in order to reduce the reservation / consumption of resources and therefore the customer’s bill (Pay-as-you-use), we speak of elasticity.

But this elasticity assumes that :

• The start of a new instance is fast enough not to penalize the users who triggered it (one or two seconds maximum)
• That this startup does not imply a high consumption of resources

But it is exactly the opposite that happens with a program using the JVM.

It can take 30 seconds (or much more) to start up, during which time it will consume a phenomenal amount of CPU cycles and GB of RAM.

In the infra world, it is said that the JVM “heats up”, like an old diesel engine.

It is therefore obvious that JVM-based technologies (Java, Scala, Kotlin, Groovy, JRuby…) are not the ideal candidates for the new Cloud paradigm… despite their hyper-presence in the enterprise application landscape.

Rewriting applications with technologies that finally prioritize performance and multithreaded processing will take time, years, probably decades, and we call this (among other things): technical debt.

A new lease on life

Rewriting (or writing) enterprise software with modern technologies, better suited to modern hardware and deployment paradigms, assumes that these technologies exist.

They do. For the last ten years, we have seen the emergence of new languages, which try to reconcile, as much as possible, productivity and performance.

Go and Rust are the two most significant representatives.

They were both designed with the new computing landscape in mind: massively multi-core architectures and a deployment paradigm based on elasticity.

They are both strongly typed, and compiled for real CPUs aiming at a level of performance and efficiency unattainable for an interpreted or semi-compiled language like Java.

They both excel in the management of concurrent programming, security performance and the reasoned but efficient consumption of hardware resources.

Productivity and performance being rather antinomic, they deal with the limits of the exercise with a different approach:

• When the limits are reached, Go focuses on simplicity and productivity.
• When the limits are reached, Rust favors performance and security.

Their popularity and adoption are seriously starting to take off, but it will be a long way before they take the place of Java and Python in the enterprise.

Personally, I’m glad they’re available and I call on all developers to take an interest in them: for the planet, and all managers and decision-makers: for their operating accounts.

The promise finally delivered

I won’t go into comparative performance and consumption figures for this or that implementation of a program in the various languages, as the web is full of them.

I will simply confirm that my personal experience attests to differences that are obvious and sufficiently important to justify in many cases the financing of the rewriting of a number of enterprise programs with these modern languages.

The financial gain in “Run” costs (i.e. the cloud provider’s bill) often quickly absorbs the investment required to rewrite the software.

Moreover, it is not excluded that on this occasion, the latter gets rid of a certain number of historical bugs by taking advantage of the new security and quality control mechanisms (e.g.: integrated unit test mechanisms, automatic or secured memory management, simplified but efficient concurrent programming models …) of these languages.

The promise of the Cloud, namely a decrease in infrastructure costs, and IT in general, has proven to be a mirage for many companies that, all things considered, struggle to justify their switch on a purely financial angle.

Other supposed advantages such as agility, risk control and my favorite one of all: the refocusing on the core business of the company, are often put forward ;-)

However, the reality is not so far from the promise:

• By pooling infrastructures in giant ultra-optimized datacenters, we effectively lower the average consumption per company.
• We allow them to pay for resources only when they are used.
• They have access to a flexibility and elasticity of processing capacities that were previously unimaginable.

But to achieve this reality, it is imperative that their development teams (or editors) dig up a skill that has often been forgotten for decades in business computing: the search for software performance and efficiency.

Conclusion

It seems that our industry evolves in cycles of about two decades.

Mainframes ruled the world between the 70’s and 90’s, micro had its heyday after about 15 years and then the Cloud paradigm appeared in 2005, it took 5 years to really break through to become the default standard by 2010.

We can assume that it will remain so at least until 2030.

Investing in performance, elasticity and software quality (sometimes at the cost of a small drop in developer productivity) is therefore not an unreasonable gamble, especially since there is no reason to believe that we will see the “free” increase in performance of the 90s to 2004 in the near future.

I’m firmly convinced that it is worth it, financially, ecologically and intellectually.