Ian Malcolm on Software Development

I’ve basically decided that I really need to get torqued off in order to get motivated to write a blog. Writing is hard, and at least in my experience, a certain amount of pent up passion is required in order to get a writing project of any length off the ground. So for my fans that have been clamoring for me to write…

Here goes nothing. :-)

I’ve encountered a pretty standard theme lately. It has a few variations, but it pretty much looks like the following:

Developer 1: hey, do you think we could make X work?

Developer 2: yeah! you just hook the foo to the bar, add a little glue, shove it into a Docker container, and…yeah! this just might work!

Developer 1: cool! let’s do it!

Meanwhile, I’m sitting in the corner all like:

Of course I rant a bit. And I go sit in the corner and rock back and forth for awhile. This situation repeats several times over a few weeks, and all of a sudden, I have a moment of clarity. We’ve been down this road before!

Welcome to…JURASSIC PARK!

It was as if he was speaking directly to me. Ian Malcolm. My old friend and chaotician. He’d warned us about exactly this. Listen to him as he pours forth wisdom:

Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.

Go ahead and substitute “programmers/developers/engineers” for “scientists” in that quote, and you’ll immediately understand my pain.

You see, I was once one of these developers. I began my career completely preoccupied with getting things to work. I believed that the computer was there to do my bidding, and that whatever I could compel, coerce, or cajole it to do was a perfectly good thing.

And then I encountered users. And expectations. And changing requirements. And bugs. And side effects. And…reality.

And one day, when I’d started to accumulate more grey hair than I’m comfortable with admitting, I realized that there is a gigantic set of things that I can make work that I shouldn’t make work. Why? Because they make dealing with the reality of software that is required to meet the needs of a large population of human beings with high and continually evolving expectations much more difficult.

In fact, and this is going to blow your mind, there is a whole class of things that requires genius level programming prowess, a sprinkling of cleverness, and a dash of luck to pull off, and you should never do any of those things ever. Because you’ll never be able to maintain it. You’ll never be able to change it. You’ll never be able to make it behave predictably. You won’t be able to summon the heroic effort required to keep it alive while you do the rest of the perfectly reasonable tasks that the business continues to place before you.

But wait. Malcolm isn’t done:

The complete lack of humility for nature that’s being displayed here is staggering.

Humility is not often cited as one of the traits that we desire in programmers, is it? We want ROCK STAR 10X FULL STACK DEVELOPERS HOPPED UP ON RED BULL! Don’t we? If you’re in doubt, hop on over to Google and search for “rock star developer wanted.” Sift through the results. And weep. Then continue reading.

I find it amazing that the average developer today (side note: by average, I mean the pattern that I detect through my regular exposure to developers from multiple companies and multiple geographies as I do consulting work, training workshops, and conference speaking) has very little exposure to any of the “classical literature” of software engineering. I routinely ask folks if they’ve read any Fred Brooks or The Pragmatic Programmer. I’d estimate 20% or less ever answer in the affirmative.

And so what do we do as an industry? It seems like we discover and rediscover the underlying principles of being good at software over and over and over again. You can see it in the evolution of architecture and design practice (OO, SOA, microservices, …), programming language communities (Java, Ruby, Node.js, …), and process (waterfall, agile, lean, …). We get into the business of developing real software with real users solving real problems, and we discover that it’s hard. And we need structure, and guardrails, and discipline. And we create things like design patterns and architecture frameworks. And things cease to be fun. That practice of compiling, coercing, and cajoling the computer to do our bidding seems like a distant memory. And so what do we do? We abandon our current world for the grass that looks ever so greener on the other side. And programming becomes fun again. And then we start again to develop real software with real users solving real problems. And…we discover that our new sandbox, complete with its abundance of joy, is somehow not up to the task. Maybe we need a design pattern here. Maybe we need some guardrails over there. And like the frog gradually being boiled to death, we recreate that world we once knew.

A few of us wake up and realize IT’S GETTING HOT IN HERE. This isn’t the world we came over here to enjoy! We’ve recreated that which we despise, that which drove us away to discover what was once our paradise. And what do we do? Do we gain the understanding that there’s a certain amount of discipline needed? A certain amount of respect? A certain amount of humility toward those who came before us and built incredible things that have literally run the world for decades?

No.

There’s a new community emerging over there. Without design patterns. Without guardrails. The grass is so green. It’s the greenest grass I’ve ever seen. Let’s go.

I’ll tell you the problem with the scientific power that you’re using here: it didn’t require any discipline to attain it. You read what others had done and you took the next step. You didn’t earn the knowledge for yourselves, so you don’t take any responsibility for it. You stood on the shoulders of geniuses to accomplish something as fast as you could and before you even knew what you had you patented it and packaged it and slapped it on a plastic lunchbox, and now you’re selling it, you want to sell it.

We’re pretty obsessed with speed these days aren’t we? I walk around with a slide in several of my decks that quotes one of my favorites, Adrian Cockcroft:

Speed wins in the marketplace.

If you’re not shipping, then someone else is. This much is true. And ultimately, it’s become required of us in this age of “digital transformation” that we move faster and faster.

And yet, I can’t help but return to this rant from Dr. Malcolm. Where is the discipline? Where is the striving for knowledge? Where is the personal responsibility taken for delivering a solution to a problem that runs, runs well, and keeps on running?

Those who came before us have made things so much easier for us developing software in 2016. We have incredible infrastructure, platforms, languages, frameworks, and libraries from which we can build things the average, mainstream software developer never would have dreamed of building even a decade ago. But I’ll tell you this: these things weren’t slapped together overnight. Smart people thought long and hard about what they were building and why they were building it before they ever arrived at how. Getting it to work was important, but getting the right “it” was even more important.

I’m not saying you don’t iterate. I’m not saying you don’t start small, get fast feedback, course correct, rinse and repeat. Of course you do. That process works.

What I’m saying is that there is a whole set of things that you shouldn’t even attempt because those who came before us have been there and earned the t-shirt to prove it. And once you eliminate those things, you still must apply the same thoughtful discipline, knowledge, humility, and responsibility to developing your own solutions. No abstraction, no matter how elegant, can compensate for just making something work without considering the long-term consequences.

Please, for the sake of all of us, before considering whether or not you can, stop and consider whether or not you should. Peace.