The Crazy and Wonderful State of Web Development
A brief look at where we’ve been, where we‘re going, and how to stay sane along the way.
One of my students pointed me towards an article on Medium that lamented the complex and fast-paced progress in web development. While I can certainly understand the frustration of that author (a frustration I’ve heard echoed by many others), I want to offer a different perspective.
As you read this article, I want you to realize two things. First, even though I’m writing about these things linearly, most of these developments were happening simultaneously. Second, this isn’t meant to be an exhaustive treatment of the history of web development technology. My goal in writing this was to help illustrate both how far we’ve come, and far we still have to go.
If I left out your favorite technology, please give it a shout out in the comments.
In the Beginning
Most of us saw the sense in div-based design, but while it was a fantastic idea, the patchy browser support for CSS made it impossible to rely on for anything but trivial styling.
So we were forced to continue using tabular layouts, even though we all pretty much agreed that they were suboptimal. So we continued to slice up our images into 9-parts, adjusted the cell spacing, and possibly adding some transparent images in various cells, in order to prevent certain browsers from collapsing them.
But, even though it would take hours to handcraft the simplest designs and make sure they worked decently well across all major browsers, we were happy because we were creating things that anyone on the planet could see.
The Demand for Interactivity
Browser-hosted application frameworks like Flash and Java had begun to rise in popularity. Suddenly you could develop a full-fledged, interactive application or game, and then deploy it over the web. We even had something called Shockwave, though many of us couldn’t keep the differences between Shockwave and Flash straight in our minds.
This was pretty exciting stuff. The only trouble was, you had to make sure the user’s browser supported the necessary 3rd party plug-ins to make this technology work, and that they had installed the right versions of those plug-ins.
But, that was okay, because now, instead of just having tinny-sounding midi files playing on our Geocities sites, we could have real audio and video files. Suddenly, everyone in the world could watch creepy videos of dancing babies!
Using Flash meant working with ActionScript, while Java was its own language, one that seemed to be obsessed with patterns of use that seemed new and foreign to many of us. Microsoft offered a proprietary solution using DHTML and JScript, and since they had the most dominant browser, many companies opted for this technology, leading to the proliferation of the popular error message, “This site has been designed to work for Internet Explorer”.
But even though we had to fight with new, proprietary languages, deal with plugin versioning conflicts and security restrictions, and make tough decisions about browser support, our websites were more interesting than ever.
Talking to Servers
Meanwhile, there was a growing demand to let our websites pull information from the server dynamically, which usually involved communication with a database. As demand for this type of interaction grew, things really began to fracture.
Most of us were tired of dealing with perl-based cgi-bin scripts by this point, and so we were all happy to see some innovation on this front. The major players in this area were ASP, PHP, JSP, and ColdFusion. All of these technologies followed the same basic model of allowing you to embed database query and processing logic right in the page. After the client requested the page, the logic would be executed, the data populated, and the page served.
But there were problems…so many problems. First, choosing a vendor tied you not just to that technology, but also dictated which web server and database technologies were available to you. Second, we suddenly had to deal with things like web security, caching, load-balancing, and a dozen other things none of us had ever thought much about before.
But even though we were dealing with even more proprietary technologies and the headaches that came with them, now the data we used to have to painstakingly handcode into HTML was being pulled from the server dynamically!
CSS Strikes Back
While all of this was happening, browser vendors were listening to the pleading of web developers to make it easier to style their pages. Cross-browser support for CSS was finally at a state where it was viable to use.
As new browsers came into being, one of the things they often promised was greater support for web standards like CSS. Now, everybody who was anybody was separating their design from their markup. CSS ZenGarden came on the scene, showing us ways to push the boundaries of what CSS offered.
We still had cross-browser support issues, and dealing with relative and fixed positioning was often painful, but design changes became easier to manage than ever before.
While all of this was going on, people were trying to decide the best way to transfer large amounts of data. XML had been around for a couple of years, but there was some general confusion about what it was. Books like “Programming XML” helped to spread the confusion about the purpose of this new technology.
But, SOAP and XML-RPC were becoming the new way to send data around the web, so it became important to understand this format and how to use it. So, we had to deal with not just XML but, XML namespaces and DTDs, and learning to turn XML into HTML using XSLT and XPath, and suddenly it was X-this and X-that and before we knew it, XML was everywhere.
Now we had the ability to transfer a hundred bytes of data with just a few kilobytes of markup and a few more kilobytes of post-processing scripts and data queries, but we had structured data locked into a predictable, parsable format, so we were pretty happy with that.
While all of that was going on, everyone started to buzz about something called AJAX, a way to communicate with the server asynchronously without having to reload the entire page.
Then people started wondering if there was a better alternative to XML, and JSON began to rise in popularity. It was lean and worked great with the RESTful web services that everyone was discussing as an alternative to SOAP.
Yes, we still had major issues with cross-browser support, and our web services now had to emit multiple data formats, but our webpages were so much more responsive, and we could connect remote data stores in new and exciting ways.
We suddenly had access to a dynamic scripting language that worked pretty well on every browser, which meant we could develop real web apps used by everyone, but nobody was completely sure how they should be architected.
Suddenly, things we used to write as ad-hoc, almost throw-away scripts had to be considered in light of the larger architecture. This required learning about and adopting new patterns. But, as we adopted these ideas, our code became more modular and maintainable in the long run.
The Rise of the Web Frameworks
There were tons of choices before us now, loads of frameworks to explore, all of which approached web app architecture in slightly different ways. The number of possible ways to structure an app became staggering, but our web apps were better than ever, and the frameworks abstracted away so many of the cross-browser nightmares we’d been dealing with for so long.
Making Things Look Better
Meanwhile, browser-based visualization technologies like SVG had gained in popularity, allowing things like the amazing D3 framework. Eventually, HTML5 and CSS3 had reached a high enough adoption level that we could create beautiful interactive visualizations without using plug-in based tools like Flash or Java.
As mobile devices became one of the major clients of websites, new responsive frameworks like Foundation and Bootstrap emerged to help us deal with issues like “mobile-first web design”. We’ve went from a deluge of Times New Roman and Arial-based text, to a huge array of typography choices thanks to resources like Google Fonts, TypeKit, FontAwesome and GlyphIcons.
As people used these various frameworks and combinations of frameworks, they began to run into things they didn’t like. Shortcomings in the abstractions, tradeoffs between convention vs configuration, and architectural constraints they didn’t want to deal with.
Then people wanted to bring some of the best ideas from functional programming into the mix, and frameworks like Immutable and Redux were created and began to gain in popularity. This new generation of frameworks gave rise to the MERN stack as a replacement for MEAN, and on and on and on…
Today it seems like there are new frameworks appearing every week, along with an accompanying set of new design patterns. Every few months, new tools like Gulp, Grunt, NPM, and Webpack emerge to help us manage the complexity of our development cycle.
We have amazing frameworks like socket.io that allow us to use advanced technologies like websockets, even where they haven’t been fully implemented by browser vendors.
Advice for Staying Sane
And so it goes, on and on. Five years from now, people will look at React and Node and Sass and chuckle at the “good old days” as they work on web development in whatever new and exciting form it has taken.
So yes, there is complexity, and sometimes the complexity can seem staggering and overwhelming. But here are a few quick tips I use to stay sane:
Remember where we’ve come from, because that puts the current state of the art in the proper perspective.
Why are we so excited about React? Because of where we were with Angular. Why were we excited about Angular? Because of where we were with the previous generation of dynamic DOM manipulation tools.
You don’t have to learn every single new framework that exists in the community.
Don’t feel like you have to be what Jeff Atwood refers to as a “Magpie Developer”. I never looked at Meteor JS, because I never needed to. I used Ember quite a bit, but didn’t use Angular very much. I never used backbone.js at all.
If you’re going to chase frameworks, focus on major paradigm shifts.
Angular/Ember/Other MVC frameworks were all basically doing different versions of the same thing. The same could be said for ASP/PHP/JSP/ColdFusion. But just as Angular and friends were big paradigm shifts compared to developing with PHP, so React is a big paradigm shift compared to Angular.
Don’t rewrite your old stuff just for the sake of rewriting it.
My favorite development stack right now is Node + React + Redux + Immutable. But I have plenty of PHP-based projects that I have no plan to rewrite in that stack.
If your goal is to gain experience or try out a new technology, then you should absolutely use the technology to create a new project, or port a small project to it.
Don’t think for a minute that rewriting your “legacy” Angular code using React is going to somehow help you gain more customers or dramatically increase your profit margins.
There are plenty of major companies still doing quite well with the LAMP stack.
Your customers don’t care what framework you use.
When people go to your site to read your content or purchase your product, they don’t care if you’re using immutable data structures, a CSS preprocessor, or a proper MVC architecture. For all they know, you’ve got a team of people in a basement somewhere, manually updating <font> tags on all of your pages.
VC’s don’t care which framework you use.
If a major part of your VC elevator pitch centers around the technology stack you’re using, you’ve probably already lost.
Web development trends seem to come and go at breakneck speed. When you’re feeling overwhelmed by the latest framework of the week, take a deep breath, and look at it objectively. If it looks like a major paradigm shift that excites you, jump in and explore it. If not, ignore it for now, and wait for the next one.
Remember why you got into web development in the first place. Remember how far we’ve come, and how far we still have to go.
As I mentioned at the start, if I left out your favorite technology or web dev headache, please give it a shout out in the comments.