The Quiet Crisis unfolding in Software Development
Bill Jordan
3.4K159

This is a nice article, but I think the title is misleading, which I’ll talk about in a minute.

Overall, it’s got some pretty sound management advice for mid to large companies. The one place where I’d disagree with it, assuming you accept it on its own terms, which I don’t (but, again, more on that soon), is the idea that “time to market” is only something that startups should care about. Even inside a company, delivering features quickly, which is the same essential idea as being a startup, even if it’s only internally, is critical. And, on this metric, most of companies suck. Speed should be a top priority for every engineering team, whether you’re a startup or a fusty old enterprise company.

Ok, having said that, if you’ll indulge this small rant, I think the title for is off track. I don’t think there is a “crisis” happening, although there is definitely something big. Second, I don’t think it’s at all quiet.

On the first point: we’re nearing the end of an old model and at the start of a big new wave. We can date the old model to November 7, 2005, with David Heinemeier Hansson’s intro to Rails ideo: https://www.youtube.com/watch?v=Gzj723LkRJY. This video was a HUGE event, and launched the technical foment of the Web 2.0 period. It was the start of a New Thing.

I remember clearly the griping from the Java community, who pointed out all the deficiencies of the Rails approach, how duplicative it was (“Why reinvent the wheel — Java can do that!”) and how it ignored all the hard problems that the Java community had solved through hard experience.

But, hey, it was the new thing, it pointed to a different way, and it was the perfect tool to take advantage of the LAMP stack and the emerging low cost infrastructure being made possible through virtual machines (VMs.). So, the Java people all wound up sounding like a bunch of “Get off my lawn, you damn kids!” fuddie-duddies, while the Rails people were the cool new thing.

But, over time, all those things the Java people were talking about — security, scalability, software processes, change control, etc — emerged as central issues, and the beautiful clarity of Rails got bogged down by all the ugly corner cases, features, and cruft that happen in “real applications” made by “real developers.” In other words, it (and its many followers in other languages) began to “mature” into today’s hated Monolith.

I think we’re at a similar point, now, as we were back in 2005, with systems like Rails being replaced by new approaches like microservices, and there is a similar backlash brewing from the new “fuddie duddies”: the very same people who displaced the old Java people. See this as an example of their line of thought: https://circleci.com/blog/its-the-future/ . (Also, see their thoughtful followup: https://circleci.com/blog/it-really-is-the-future/.

This time, the driving force is Docker, which replaces a VM with an even simpler container/process model, and — much more fundamentally — the move towards treating infrastructure as ephemeral functions that exist only for a few milliseconds. This is best represented at scale by services like AWS Lambda, which stress functional computing and immutable models, and new frameworks like http://serverless.com/.

So, this is where my objection to the term “crisis” comes in. There is not a crisis: there’s just a new wave in town that’s driven by the same compelling economics of falling costs. We’ve gone from thousands of dollars (the old SUN Microsystems + Oracle stack + Java) to hundreds of dollars (Rails on various VMs) to now tens of dollars (computing as lambda functions) or even pennies. Each of these models usher in a set of new tools that displace the old, and things suck until it all inevitably shakes out. Then it all begins again. Thanks a lot, Gordon Moore!

Now, to the second point: about this being quiet. I actually think there is a clamor around any number of technical crises (as opposed to managerial crises). Among them:

* The emergence of design as a competitive advantage, and how crappily the industry is responding to producing more designer+developer “unicorns.”

* The framework churn on the frontend, most succinctly described by https://medium.com/@ericclemmons/javascript-fatigue-48d4011b6fc4#.ehhsbnac1. Love it or hate it, Javascript is probably on of the most important languages on the planet, and it’s a total mess.

* The shift to new UIs and UX that go beyond screens and into things like voice (think Alexa), and the desperate need for “mainstream AI” to power these new interfaces.

* The accelerating fragmentation of the development landscape, in general. We’ve gone from a few big platform providers (Microsoft, etc) that used to produce tools on a stable schedule to a hodgepodge of open source that creates environments so brittle that one person changing a library that formats a string brings total and complete chaos. (http://www.businessinsider.com/npm-left-pad-controversy-explained-2016-3). Personally, I think the tradeoff is more than worth it, but it certainly creates a lot of churn and real pain.

Those (particularly the first and the last), strike me as the real crises, and there are a lot of people talking about them!

It’s not at all “quiet” out there.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.