2017: The Year of cgi-bin…er, Serverless

It’s easy to dismiss a new trend in technology by simply pointing out that the concept has been tried before, or that it is a re-hash of an older technology.

You might think, given the headline, my intent is to dismiss “Serverless” for not being new enough, but not so! I’m not mocking new inventions for re-inventing earlier inventions - as Greenspun’s tenth rule of programming states:

Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified bug-ridden slow implementation of half of Common Lisp.

Let’s look at what made cgi-bin a success story of the early web in the first place, and why it’s time for a resurgence of its initial intentions.

What is a cgi-bin anyway?

When I started to learn about developing web applications in the late ’90s, the cgi-bin wasn’t just established technology, it was already crossing in to the working-but-uncool territory. (Cool technology, at the time, was the marquee HTML tag).

The CGI (Common Gateway Interface) was introduced in 1993 to enable the execution of scripts by a web server. A user would request a URL like http://www.example.com/cgi-bin/hello.pl and the web server (on example.com, per this example) would locate and execute the hello.pl script, passing request parameters as environment variables and returning the stout (standard output) in the body of the response.

CGI bridged the world of the web and the world of UNIX, enabling the creation of true web applications.

What was so great about cgi-bin?

Without CGI, there would have been no web applications. If you look closely at the URLs of some early web apps like Yahoo or eBay, you will still see the letters cgi-bin. Most likely, the applications are powered by other technologies by now, and the URLs are relics, pointing to the technological underpinnings of a bygone era.

CGI was deceivingly easy to develop: just drop a script into the cgi-bin directory and you were up and running. The “just” hid a number of challenges for the aspiring 1993 developer, including getting access to a UNIX machine, knowing how to code in strange languages like Perl, and understanding how to set permissions to make sure the script was actually executed. CGI made web development available to a new class of developers.

Maybe most importantly, CGI opened the door for innovation. Technologies like PHP, now one of the top 5 programming languages, started as little more than a CGI-script. Many of the most powerful technologies start as little more than interesting toys, before they evolve into more versatile, secure and scalable versions of themselves.

Why did the web move on from cgi-bin?

Despite it’s continuing popularity in URLs (!), CGI today is essentially a dead technology. In 2016, it look like there has been about 400 questions regarding CGI on Stack Overflow, while there has been roughly 340X as many questions for PHP.

When you look at the reasons why the web moved on, three stick out: security, performance and the ability to absorb complexity.

Performance and security issues are easily explained: each CGI script or program would run in its own process, as a program of its own. This would add performance overhead because for each request, the script would need to get parsed, the interpreter launched, the script executed, etc. Also, each CGI script was a full-fledged program that had all privileges of the parent process (usually the web server).

The cgi-bin had a drastic combination: get started easily (just drop a script at the right location) and suddenly get the power of unlimited system access. As we say, with great power, comes great responsibility. Unfortunately, many developers didn’t act - and maybe didn’t even know how to act - responsibly. This led to insecure and error-prone systems.

CGI’s lack of an ability to absorb complexity means that it was easy to get started building a CGI application, but hard to build robust, scalable complex applications. While techniques for managing modularity of applications were present in programming languages at the time, they were not popular in the scripting languages used with CGI.

How is serverless more than cgi-bin, reborn?

While it’s easy to frame new serverless technology in terms of existing concepts, the serverless runtimes that entered the market in 2015 and 2016 are markedly better than the old cgi-bin approach in 3 ways:

  1. Serverless runtimes provide a much higher level of isolation due to the combination of containers at the operating system level and virtual machines at the runtime level. If you write a serverless function in Javascript and want to cause harm, it would need to break out of the Javascript virtual machine (node.js and V8) and out of the Docker container.
  2. The performance problems of the cgi-bin were mostly scalability problems. While the overhead of developing a new process is relatively small, when the server has to process thousands of requests per second, it becomes notable. Even worse, in 1993 you did not have other servers to delegate your workload to. Thankfully, in 2017, computing power is available in unlimited quantity to everyone with a credit card. Serverless runtimes solve the scalability problem not by eliminating overhead (you could argue, there is more overhead, albeit on faster machines, with multiple levels of containers, and request processing that is distributed among machines, not just processes), but by allowing to scale out horizontally.
  3. Now we are better prepared to deal with the complexity impact of more powerful applications. Serverless programming goes hand in hand with functional programming, and allows developers to decompose applications into small, stateless functions instead of complex, stateful object graphs. As developers, we have learned to take our APIs seriously, and treat each API as a microservice of its own, and we use continuous deployment where two decades ago FTPing the Perl file has been good enough.

Finally, while the cgi-bin has been the first web-native technology, Serverless is the first truly cloud-native technology that allows developers to forget about the servers of the past.

What makes 2017 the year of Serverless?

In 2015 the first Serverless runtimes made it to market. In 2016, the concept got traction among early adopters and innovators. For example, IBM’s OpenWhisk entered the Apache Incubator, making it the first serverless runtime that is not just open source, but also under open governance.

In 2017, we will see how Serverless is driving the creation of new types of applications: hybrid and progressive web applications, personal voice assistants and messaging bots.

In 2017, we can also expect the ease of serverless development to increase, not just with costs per millisecond approximating zero, but also with reduced overhead for administration, and even development. In 1993, all you needed was a UNIX box. In 2017, all you need is a touch screen and wifi (which is, if you look closely enough, just another UNIX box…but that’s another blog post).

In 2017, ubiquitous serverless runtimes will be the catalyst for innovations in application frameworks that use functional composition to create more complex and powerful serverless applications for creatives, marketers and knowledge workers.

In 2017, expect to hear more - not less - about Serverless.

Celebrating the new year by blowing up some servers.