FROM THE ARCHIVES OF PRAGPUB MAGAZINE MARCH 2015
Patterns Enhance Craft: Thoughts on Patterns
By Kent Beck
Kent has been through the patterns-vs-craft fight and come out on the other side.
Kent has been publishing his thoughts about patterns on Facebook recently, and he has been gracious enough to allow us to pull them together into one essay and share it with you here.
Step 1: Programs Is Programs
Until this post I had no idea of the depth of feeling behind the contemporary pattern backlash. One phrase in particular struck me.
Patterns is one of the most powerful ideas we have. Critics may be right that it devalues the craft, but we would all do well to remember that the craft of software is a means to an end, not an end.
This quote took me back to my early days with patterns, when I too struggled with the impact of patterns on craft. This article describes my struggle with the apparent contradiction between patterns and craft, and how I resolved it to my satisfaction.
Internal Forces
One of the insights I learned from Notes on the Synthesis of Form is that the forces shaping most design decisions are generated internal to the process of design, not by external constraints. I hadn’t really thought about it before, but I would have guessed that most decisions were shaped by what an artifact needed to do, not what it was. An example.
Say we are building an airport terminal. The forces shaping our first decisions are external: so many people and planes per year, budget of thus and so, so many airlines, a contracting process like this. After those first decisions have been made, though, the decisions are quickly influenced mostly by the fact that we are building something — a roof has to support a snow load and a wind load and transfers forces to the walls in certain ways. These forces are internally generated. They have nothing to do with whether the building is financed privately or with a municipal bond.
Now we are designing a barn. This seems like a completely different problem — different budget, different size, different setting, different financing. At first, it is a different problem. We figure out how many of which kinds of animals to shelter, where to put milk storage, and how to provide a separate entrance to the cheese gallery. As the scale of decisions decreases, though, we have to figure out the roof. The list of forces influencing the roofing decision is the same list of forces we encountered in the airport terminal — snow, wind, and so on — although their relative strengths may be different. Again, though, the forces are internally generated.
Turning to programming, I work on software that runs in a carefully-controlled data center, where CAPEX and OPEX are primary concerns and other programs are the consumers of results. I also work on software that runs on hundreds of millions of handsets and browsers, where a person consumes the results and cares about latency and battery life. Different external forces, but most programming decisions are made in response to internally generated forces:
- Programs use the same bits of logic repeatedly
- Programs are modified many times
- Programs consume most of their resources in a small percentage of the code
- People work with programs
These are the forces that programmers resolve many times a day, forces that arise simply because we reason about an uncertain future with a fallible brain.
The next insight takes this argument a step further, to notice that programming problems recur.
Step 2: Programming Rhymes
The goal of this article is to establish an argument that patterns enhance the craft of programming, that patterns don’t devalue craft. I’m qualified to make this argument because I went through my own “dark night of the pattern soul” some time back and came out the other side convinced that my craft was nourished by understanding and using patterns.
Step 1 established that the forces influencing most software design decisions are generated by the act of building software. When I am writing a loop, it mostly doesn’t matter if I am building a missile nose cone or a social network. The conflicting constraints I resolve are created because I am programming, not by the external problem I solve.
The second step in arguing for the importance of patterns to craft is to note that the problems created by these forces look similar. Take variable naming, for example. The reader of a variable needs to understand:
- Scope
- Lifetime
- Type
- Purpose
Variable length is also constrained — too short and they don’t contain enough information and too long and they take too long to read and are difficult to format. When I name a variable, I balance these considerations.
Every time I name a variable this same configuration of constraints appears. The scope, lifetime, type, and purpose may be different from one variable to the next, but the essence of the variable naming problem is the same. That is what I mean when I say that programming rhymes. There is something different about each variable, which is how we come up with different names, but there is something similar about all variable naming decisions.
Imagine that you had a catalog of these configurations of constraints that everyone encounters all the time. Naming, computing with collections, computing over recursive data structures, differentiating normal and exceptional control flows, and on and on. What would that look like? How different would it be between languages? How different would it be between programmers? What percentage of daily programming decisions would be covered?
Sometimes I come upon a truly novel problem while programming, or I get an insight for a truly novel approach to an existing problem. That does happen, and it is critical to my conclusion that patterns enhance craft. However, the majority of decisions I make are of a form (configuration of constraints) that I have encountered before.
Before moving on to Step 3: A Few Good Solutions, I’ll take a side trip into my own concerns about patterns and craft.
Sidebar: My Personal Crisis
My history with patterns began when I chanced on The Timeless Way of Building in the University of Oregon bookstore. As a poor undergraduate I couldn’t afford to buy it, but I was fascinated. I ended up reading it standing up in the store a half hour at a time.
After I joined Tektronix, I was walking by Powell’s one day and saw a copy of Notes on the Synthesis of Form in the window. I knew I was interested in Christopher Alexander, so I went in and bought it (actually I bought both copies they had and ended up giving the other one away years later, but
that’s a different story). I was hooked on Synthesis of Form when I read
the Poincaré quote in the preface: “Sociologists discuss sociological methods; physicists discuss physics.” I wanted a way to talk about programs.
When I first started to program all was chaos. Quickly, though, patterns
began to emerge. If/then/else was the first one I “discovered,” synthesized
out of conditional and unconditional gotos. I loved the feeling of taming
chaos with the power of my mind.
By the time I had been programming for a year at Tek Labs I was running
into the limitations of my brain. I could write complicated programs but I couldn’t keep them running when I tried to incorporate the excellent suggestions of my colleagues. I couldn’t explain my programs (Sam Adams
used some of my code from that era to encourage students that they too could get over complexity). I still loved the feeling of inventing mechanisms that uncovered hidden simplicity, though.
When Ward Cunningham and I started discussing patterns for programming, it became clear that embracing them would change my emotional and intellectual experience of programming. If I accepted the premises that forces are internally generated, that configurations of forces recur, and that a few solution “shapes” dominate, then what becomes of my self-image as a Staff of Truth wielding wizard? As someone who battled all my life to establish my own identity, this was terrifying. If I wasn’t going to be a wizard post-patterns, what was I going to be? A lego-plugging drone,
doomed to forever assemble pieces designed by the real heros?
That was my patterns versus craft crisis. How I passed through the door and what lay on the other side are the subjects of the rest of this article.
Step 3: A Few Good Solutions
So far we have internally generated forces in repeating configurations of problems. The next step in establishing the relationship between patterns and craft is to observe that for any given repeating configuration of forces, there are only a handful of “solution families.” That is, there are a few good solutions and an infinite universe of bad solutions.
I tried to come up with a logical argument for why this is and I failed. Something something mumble fractal mumble attractor something local maxima. That’s about as far as I got. However, my experience suggests that the observation is valid, so I’m going ahead with it.
Why isn’t there one solution, as Taylor would have us believe? Because while the configuration of forces may repeat, their relative strengths do not. For example, Martin Sústrik in a critique of C++ for ZeroMQ emphasizes that while exceptions are good for preventing a program from failing, they are not good for eliminating undefined behavior. Most programs aren’t sensitive to a little non-determinism. For those, exceptions are fine. Amplify the “must be deterministic” force, though, and exceptions are dominated by “return value plus errno.”
All of the common styles of error handling are appropriate for different relative weights of constraints when calling an unreliable routine. Depending on whether readability, reliability, automated analysis, performance, or future maintenance is most important you could reasonably choose any one of:
• Exceptions
• Return value plus errno
• Exceptional value (e.g., Haskell’s Maybe)
• Success and failure callbacks
One challenge skilled-but-inexperienced developers face is that they have been successful with one style of solutions and one relative weighting of forces.
If you’ve never seen a screw, it looks a lot like a nail. If you have a hammer in hand, the result is, well, a learning opportunity. Some of the best programmers I’ve worked with have the habit of overcoming this bias by overusing any new technique until they get a feeling for the forces involved.
The existence of a proven approach does not mean that a problem is already solved. Subtle shifts in the forces influencing a decision result in subtle shifts in the solution, shifts that must be creatively interpreted by a person. Even a solution as fixed as “call this library function” requires understanding the parameters to be passed, correctly interpreting the results, handling error conditions, and becoming aware of the consumption of limited shared resources like memory or power.
Just because there are already a set of proven approaches to a given problem does not mean that a new, competitive approach is impossible, just that it is unlikely. The most common case is when the forces have shifted without anyone noticing. Along comes a newcomer, says “this makes no sense,” and invents a new approach. Maybe they are seeing new or reprioritized forces that everyone else is blind to through familiarity, or maybe they just don’t understand the problem yet. Still, “Why is this any harder than X?” is the single question that has brought me the most satisfaction.
Now the table is set to answer the original questions, “What is the relationship between patterns and craft? Does applying patterns really turn me into a mindless drone?”
The Final Step, Pareto Effort
After I had been programming professionally for eight or nine years I grew discontented with my skills. I suspected that I had another gear, but I wasn’t going to find it through incremental improvement. I decided to learn programming from scratch, starting by learning to properly touch type.
When I started programming again, I vowed not to type a single character unless I knew what pattern I was applying as I did so. The result was incredibly frustrating. I want a class called “Stack,” but why “Stack” and not something else? Then I would go and write the patterns for naming classes and then I could type “Stack.” Then I would want to make its first method public, but why public?
At first, it was like a centipede thinking about its feet. Programming, which had been effortless and flowing, was now an awful chore. I knew, I just knew, that I was going to encounter more patterns I hadn’t thought about. Decisions I had been making on auto-pilot I would have to stop and consider. Ten seconds of coding, four hours of pattern writing.
By the end of the first week, though, I discovered I wasn’t finding many unexamined decisions any more. Most of the decisions I made most of the time were covered by a small catalog of patterns, tens but not hundreds. The surprising effect was that as long as I didn’t uncover any new patterns, programming already flowed better for me. When I programmed, I programmed, undistracted by that second level of anxiety about whether I was using the right strategy.
Pareto Effort
Patterns let me craft when I craft, and talk about craft when I talk about craft. When I reach a novel situation not covered by patterns I already have, it’s clearer to me that the situation is not covered and I can step back and solve it from first principles. Solve enough similar problems and I’m ready to start thinking about patterns again.
That’s what I mean by “Pareto Effort.” When naming a variable, I need to think about the role of that variable in the logic and how to convey that to a future reader. I don’t need to think about logic gates and economics and the internal design of the code editor I’m using. I just need to put myself in the place of a future reader and figure out what that reader wishes this variable was named.
Most programming decisions are routine, in the sense that a decision like this has happened a million times today all around the world, and in the sense that the forces influencing this decision are the same everywhere (even if their relative strengths are different), and there are a handful of approaches to resolving those forces, and there is usually one approach that makes the most sense. I am not, nor are my programs, special flowers requiring a completely unique and creative approach. Empathy is hard enough for a programmer like me without piling all of economics and physics and logic on top of it. I am better served focusing my energy on the degrees of freedom that generally make a difference and ignoring the degrees of freedom that don’t.
An unexpected effect of using patterns is how much more efficient and precise technical conversations became. Disagreements became, “Accessor,” “No, Direct Access.” Those four words become shorthand for hours of philosophical discussion. I see the forces like this, you see the forces like that. At the limit, communication between programmers who deeply share a pattern vocabulary becomes grunting and pointing and gestures.
A personal effect of patterns was that they gave me a way to think about paths through the solution space. Pre-patterns I would say, “Here, this is the solution.” Someone else would say, “No, that is the solution.” The fur would begin to fly. Patterns became my “moves” in a game of design. We’re here. This pattern moves us in that direction. Then that and that and that and we’re over there. Importantly, I didn’t have to execute all those patterns this instant. Even if we disagreed on the final destination, we could agree on the next step and use that to learn about the following step.
The cost of using patterns is that I have to give up the illusion that I am infinitely creative. I don’t invent a new programming language with optimizing compiler and novel operating system and complete programming environment for every line of code. To make best use of my three billion seconds, I should ignore most of my options and focus on a few degrees of freedom that really matter right now.
That’s how patterns enhance craft: move faster, communicate clearer, think ahead productively and confidently, talk about craft clearly when the time comes to talk about craft, and save creativity for truly novel situations. It’s been worth it for me.
About Kent Beck
Kent Beck is the founder and director of Three Rivers Institute (TRI). His career has combined the practice of software development with reflection, innovation, and communication. His contributions to software development include patterns for software, the rediscovery of test-first programming, the xUnit family of developer testing tools, and Extreme Programming. He currently divides his time between writing, programming, and coaching. Beck is the author/co-author of Implementation Patterns, Extreme Programming Explained: Embrace Change 2nd Edition, Contributing to Eclipse, Test-Driven Development: By Example, Planning Extreme Programming, The Smalltalk Best Practice Patterns, and the JUnit Pocket Guide. He received his B.S. and M.S. in Computer Science from the University of Oregon.
