A brief look at where we’ve been, where we‘re going, and how to stay sane along the way.
One of my students pointed me towards an article on Medium that lamented the complex and fast-paced progress in web development. While I can certainly understand the frustration of that author (a frustration I’ve heard echoed by many others), I want to offer a different perspective.
As you read this article, I want you to realize two things. First, even though I’m writing about these things linearly, most of these developments were happening simultaneously. Second, this isn’t meant to be an exhaustive treatment of the history of web development technology. My goal in writing this was to help illustrate both how far we’ve come, and far we still have to go.
If I left out your favorite technology, please give it a shout out in the comments.
In the Beginning
I started doing web development work in the mid 90’s. CSS and JavaScript had just come onto the scene. Most people had begun to gravitate away from nested frames, and the current debates were all about table-based vs div-based layouts.
Most of us saw the sense in div-based design, but while it was a fantastic idea, the patchy browser support for CSS made it impossible to rely on for anything but trivial styling.
So we were forced to continue using tabular layouts, even though we all pretty much agreed that they were suboptimal. So we continued to slice up our images into 9-parts, adjusted the cell spacing, and possibly adding some transparent images in various cells, in order to prevent certain browsers from collapsing them.
But, even though it would take hours to handcraft the simplest designs and make sure they worked decently well across all major browsers, we were happy because we were creating things that anyone on the planet could see.
The Demand for Interactivity
Meanwhile, the desire to make web pages more interactive was growing. JavaScript had been on the scene for a while, but as with CSS, browser support was so bad that it was tough to rely on JavaScript for anything but the most trivial uses. So, we looked elsewhere for interactivity.
Browser-hosted application frameworks like Flash and Java had begun to rise in popularity. Suddenly you could develop a full-fledged, interactive application or game, and then deploy it over the web. We even had something called Shockwave, though many of us couldn’t keep the differences between Shockwave and Flash straight in our minds.
This was pretty exciting stuff. The only trouble was, you had to make sure the user’s browser supported the necessary 3rd party plug-ins to make this technology work, and that they had installed the right versions of those plug-ins.
But, that was okay, because now, instead of just having tinny-sounding midi files playing on our Geocities sites, we could have real audio and video files. Suddenly, everyone in the world could watch creepy videos of dancing babies!
Using Flash meant working with ActionScript, while Java was its own language, one that seemed to be obsessed with patterns of use that seemed new and foreign to many of us. Microsoft offered a proprietary solution using DHTML and JScript, and since they had the most dominant browser, many companies opted for this technology, leading to the proliferation of the popular error message, “This site has been designed to work for Internet Explorer”.
But even though we had to fight with new, proprietary languages, deal with plugin versioning conflicts and security restrictions, and make tough decisions about browser support, our websites were more interesting than ever.
Talking to Servers
Meanwhile, there was a growing demand to let our websites pull information from the server dynamically, which usually involved communication with a database. As demand for this type of interaction grew, things really began to fracture.
Most of us were tired of dealing with perl-based cgi-bin scripts by this point, and so we were all happy to see some innovation on this front. The major players in this area were ASP, PHP, JSP, and ColdFusion. All of these technologies followed the same basic model of allowing you to embed database query and processing logic right in the page. After the client requested the page, the logic would be executed, the data populated, and the page served.
Soon, we were moving beyond what we could do with simple cookies and having real, server-stored session state. E-commerce hit its heyday as it was suddenly easy to present a large, online catalog and deal with payment processing. Who cared if the user’s browser didn’t support some aspect of JavaScript, we’d just let the server handle it all!
But there were problems…so many problems. First, choosing a vendor tied you not just to that technology, but also dictated which web server and database technologies were available to you. Second, we suddenly had to deal with things like web security, caching, load-balancing, and a dozen other things none of us had ever thought much about before.
But even though we were dealing with even more proprietary technologies and the headaches that came with them, now the data we used to have to painstakingly handcode into HTML was being pulled from the server dynamically!
CSS Strikes Back
While all of this was happening, browser vendors were listening to the pleading of web developers to make it easier to style their pages. Cross-browser support for CSS was finally at a state where it was viable to use.
As new browsers came into being, one of the things they often promised was greater support for web standards like CSS. Now, everybody who was anybody was separating their design from their markup. CSS ZenGarden came on the scene, showing us ways to push the boundaries of what CSS offered.
We still had cross-browser support issues, and dealing with relative and fixed positioning was often painful, but design changes became easier to manage than ever before.
XML
While all of this was going on, people were trying to decide the best way to transfer large amounts of data. XML had been around for a couple of years, but there was some general confusion about what it was. Books like “Programming XML” helped to spread the confusion about the purpose of this new technology.
But, SOAP and XML-RPC were becoming the new way to send data around the web, so it became important to understand this format and how to use it. So, we had to deal with not just XML but, XML namespaces and DTDs, and learning to turn XML into HTML using XSLT and XPath, and suddenly it was X-this and X-that and before we knew it, XML was everywhere.
Now we had the ability to transfer a hundred bytes of data with just a few kilobytes of markup and a few more kilobytes of post-processing scripts and data queries, but we had structured data locked into a predictable, parsable format, so we were pretty happy with that.
AJAX and the Resurgence of JavaScript
While all of that was going on, everyone started to buzz about something called AJAX, a way to communicate with the server asynchronously without having to reload the entire page.
It used JavaScript and XML to accomplish this magic, but as with all things JavaScript, cross-browser support was still pretty spotty. So we had to write a bunch of boiler-plate code to make the request in a way that would work across all of the major browsers. But this was asynchronous data retrieval!
Who cared if it took dozens of lines of horrible-looking JavaScript to make it work. Sure, we had to deal with XML again, but we’d been down that road, we could handle the pain for the sake of asynchronous partial page updates.
Then people started wondering if there was a better alternative to XML, and JSON began to rise in popularity. It was lean and worked great with the RESTful web services that everyone was discussing as an alternative to SOAP.
Yes, we still had major issues with cross-browser support, and our web services now had to emit multiple data formats, but our webpages were so much more responsive, and we could connect remote data stores in new and exciting ways.
JavaScript Starts to Grow Up
The adoption of AJAX seemed to open a floodgate of JavaScript development. Amazing libraries like jQuery were developed to help abstract away the cross-browser issues we all hated to deal with.
Eventually, as browser-support for JavaScript became better and better, and vendors like Google put real effort into optimizing their JavaScript engines, suddenly, a new vista opened up in the world of web development: pure JavaScript.
We suddenly had access to a dynamic scripting language that worked pretty well on every browser, which meant we could develop real web apps used by everyone, but nobody was completely sure how they should be architected.
As languages evolve, community-sanctioned best-practices tend to coalesce around them. But with JavaScript, everything was happening so fast, and the language had such a long and confusing history, that people started to run in a bunch of different directions.
Eventually, a set generally accepted ideas started to emerge. Douglas Crockford wrote JavaScript the Good Parts, people began to develop utility libraries like underscore and backbone, and to migrate away from big toolsets like jQuery towards “pure JavaScript”. Even StackOverflow users had stopped saying “use jQuery” in response to every JavaScript question.
Suddenly, things we used to write as ad-hoc, almost throw-away scripts had to be considered in light of the larger architecture. This required learning about and adopting new patterns. But, as we adopted these ideas, our code became more modular and maintainable in the long run.
The Rise of the Web Frameworks
Now things began to really move forward. With the rise of resources like GitHub and GitHub Pages, it was easier than ever to create a new JavaScript framework and share it with the world.
People began to discuss the best ways to implement JavaScript for large-scale projects. People began discussing new, JavaScript-specific design patterns like IIFE, Modules, and more. Hundreds of frameworks like Ember, Meteor, Angular, and many others, rose and fell around these ideas as people looked for the best abstractions to use when structuring large-scale web apps.
The idea of using MVC and single-page apps developed, and suddenly we had a whole host of frameworks being developed to help us follow this design.
There were tons of choices before us now, loads of frameworks to explore, all of which approached web app architecture in slightly different ways. The number of possible ways to structure an app became staggering, but our web apps were better than ever, and the frameworks abstracted away so many of the cross-browser nightmares we’d been dealing with for so long.
Making Things Look Better
Meanwhile, browser-based visualization technologies like SVG had gained in popularity, allowing things like the amazing D3 framework. Eventually, HTML5 and CSS3 had reached a high enough adoption level that we could create beautiful interactive visualizations without using plug-in based tools like Flash or Java.
CSS preprocessors like Sass and Less become mainstays in web development, allowing us to optimize and modularize even our CSS files.
As mobile devices became one of the major clients of websites, new responsive frameworks like Foundation and Bootstrap emerged to help us deal with issues like “mobile-first web design”. We’ve went from a deluge of Times New Roman and Arial-based text, to a huge array of typography choices thanks to resources like Google Fonts, TypeKit, FontAwesome and GlyphIcons.
And yes, even though it seems like Flat design just won the war with Skeuomorphic, and we’re now debating Flat vs Material, that’s okay, because our websites are looking better and better.
Server-Side JavaScript
While all this that was happening, people began thinking about using JavaScript across the entire stack, not just for client-side scripting, but for the server-side as well. The appeal of having a unified language across the stack was so strong, this idea gained traction very quickly.
With the adoption of MongoDB, we suddenly had things like the MEAN stack as a replacement for LAMP, where we could use JavaScript not just on the web server and client, but in the database as well! The entire development stack could now be written in a single language!
As people used these various frameworks and combinations of frameworks, they began to run into things they didn’t like. Shortcomings in the abstractions, tradeoffs between convention vs configuration, and architectural constraints they didn’t want to deal with.
So, developers took their favorite parts of the existing frameworks and began working on the next generation, leading to the creation of frameworks like React and Angular 2.
Then people wanted to bring some of the best ideas from functional programming into the mix, and frameworks like Immutable and Redux were created and began to gain in popularity. This new generation of frameworks gave rise to the MERN stack as a replacement for MEAN, and on and on and on…
Today it seems like there are new frameworks appearing every week, along with an accompanying set of new design patterns. Every few months, new tools like Gulp, Grunt, NPM, and Webpack emerge to help us manage the complexity of our development cycle.
During this time, people were also lauding the upcoming release of ECMAScript, which was once based on JavaScript, but has advanced so much that now JavaScript is based on it. And people are so excited to use these features before they’re released and adopted by major browser vendors that they have developed frameworks like Babel, which allow you to “transpile” your ES6 code to browser-compatible JavaScript.
We have amazing frameworks like socket.io that allow us to use advanced technologies like websockets, even where they haven’t been fully implemented by browser vendors.
And then, just as you think you’ve gotten a handle on everything, people start talking about Isomorphic JavaScript, where the same code is used on the client and the server. And you’re either super excited, or you’re ready to throw in the towel and go back to C++ programming, where the biggest innovation you have to worry about is Boost.
Advice for Staying Sane
And so it goes, on and on. Five years from now, people will look at React and Node and Sass and chuckle at the “good old days” as they work on web development in whatever new and exciting form it has taken.
So yes, there is complexity, and sometimes the complexity can seem staggering and overwhelming. But here are a few quick tips I use to stay sane:
Remember where we’ve come from, because that puts the current state of the art in the proper perspective.
Why are we so excited about React? Because of where we were with Angular. Why were we excited about Angular? Because of where we were with the previous generation of dynamic DOM manipulation tools.
You don’t have to learn every single new framework that exists in the community.
Don’t feel like you have to be what Jeff Atwood refers to as a “Magpie Developer”. I never looked at Meteor JS, because I never needed to. I used Ember quite a bit, but didn’t use Angular very much. I never used backbone.js at all.
If you’re going to chase frameworks, focus on major paradigm shifts.
Angular/Ember/Other MVC frameworks were all basically doing different versions of the same thing. The same could be said for ASP/PHP/JSP/ColdFusion. But just as Angular and friends were big paradigm shifts compared to developing with PHP, so React is a big paradigm shift compared to Angular.
Don’t rewrite your old stuff just for the sake of rewriting it.
My favorite development stack right now is Node + React + Redux + Immutable. But I have plenty of PHP-based projects that I have no plan to rewrite in that stack.
If your goal is to gain experience or try out a new technology, then you should absolutely use the technology to create a new project, or port a small project to it.
Don’t think for a minute that rewriting your “legacy” Angular code using React is going to somehow help you gain more customers or dramatically increase your profit margins.
There are plenty of major companies still doing quite well with the LAMP stack.
Your customers don’t care what framework you use.
When people go to your site to read your content or purchase your product, they don’t care if you’re using immutable data structures, a CSS preprocessor, or a proper MVC architecture. For all they know, you’ve got a team of people in a basement somewhere, manually updating <font> tags on all of your pages.
VC’s don’t care which framework you use.
If a major part of your VC elevator pitch centers around the technology stack you’re using, you’ve probably already lost.
Conclusion
Web development trends seem to come and go at breakneck speed. When you’re feeling overwhelmed by the latest framework of the week, take a deep breath, and look at it objectively. If it looks like a major paradigm shift that excites you, jump in and explore it. If not, ignore it for now, and wait for the next one.
Remember why you got into web development in the first place. Remember how far we’ve come, and how far we still have to go.
As I mentioned at the start, if I left out your favorite technology or web dev headache, please give it a shout out in the comments.
