How to Make Things Better: A Case Study

Because thinking is hard
Because it’s easier than thinking, people like to implement processes or tools and then, sometimes, try to determine their purpose. The soothing siren song of the latest flow-chartable cure-all that will make everything better is just so tempting. This includes us software makers, as we are enticed by a milieu of both ideas and tools that all the cool people are apparently using at any given time. Perhaps we even peddle them ourselves.

Just how often do you plan on making sausage?

Recognizing this folly, some radical purists reflexively reject tools and ̶p̶r̶o̶c̶e̶s̶s̶e̶s̶ red tape. This is also easier than thinking. Real Agile Practitioners just use sticky notes after all.

Real cooks just use string and fire

Somewhere in the middle ground, there are ways to facilitate choosing and validating processes and tools rationally and productively. I’ll present a few here in the context of a recent effort to aid comprehension and orientation in one of our longer-term projects.

State the goal first
If you can’t say what you are trying to accomplish it’s impossible to determine if you have accomplished anything. The goal may be a problem you are trying to solve, a new benefit you are trying to achieve, or a question you are trying to answer. In my case, there are a collection of related goals:

Questions from a business perspective: Overall, what am I getting for what I’ve paid?

• Do the things that exist correlate to the things I asked for?
• Are there enough things? Are they good things?

Questions from a developer perspective: I’m building or firefighting and I’m looking at a piece of code and wondering:

• For what purpose was this code written?
• By whom?
• Was it ever deployed? If so: When? Where? Initiated how? Did it pass CI tests?

Problems noted by multiple parties: I think something is broken and:

• We need to fix it
• We want to know when, why and how it broke so that we can hopefully learn something

Start with nothing
We have all the raw information needed to answer the above questions, but we need to organize it in time. In the interest of keeping things simple, here’s a first crack at a system history organized by time:

Alternatively, here is a representation of the system history organized by relationships (i.e. a graph)

Perhaps I need a bit more detail, but together these diagrams do express the following promising directions:

• Visual arrangement is a form of information. It concisely represents concepts that humans can understand
• Linear arrangement effectively and compactly expresses the concept of time
• Graphs effectively represent the relationships between things

Between my written word goals and minimal diagrams, I have a much clearer understanding of what I actually need.

Reductio ad absurdum
A related and time-honored strategy for testing the limits of an argument is to extend it to its extremes and see what falls out. This is essentially what I did by analogy in my opening. My point was that credulously adopting every process, pattern, and tool that some “thought leader” plugs on their blog quickly leads to an absurd and incompatible overload analogous to the proliferation of kitchen gadgets that can overwhelm cooks with the means and inclination to follow every consumerist bunny trail on offer in today’s market.

Going to the other extreme likewise reveals the absurdity of saying that all tools and processes are just superfluous doodads and red tape. Doing everything with your bare hands and zen master instinct does not scale, no matter how badass it makes you feel. I will not be a more productive developer if I have to go back to programming with a hole punch. And the time required to clean my stove and wash my pans and knives is far eclipsed by the time they save me in preparing food compared to, I don’t know, roasting freshly caught squirrels on a spit over an open fire in my backyard for every meal. Although my kids would probably love to try.

In my case, one extreme would be to start ditching tools we are already using. There’s nothing to stress about if I don’t even have a Pivotal Tracker, GitHub, Travis, or Heroku to look at, right? If composing and connecting the dots is such a challenge, why not just remove the dots? Just add some commit comments and `git log` to enlightenment.

’nuff said, right?

This is where having the goals written down and at hand helps to determine if the outcome is good or absurd. Getting rid of the existing tools and processes is absurd not because it wouldn’t provide the same cathartic satisfaction as permanently deleting your entire email history, but because we haven’t met any of our stated goals. In fact, we have made it even harder to meet them.

Going the other direction could entail adopting a mega platform like the Atlassian suite, Azure Dev Ops, or Service Now. We could connect ALL the things, natively and richly. No more manual hyperlinks from pull requests comments to stories, no more tedious backtracking from cryptic commit hashes on Heroku deploys back to GitHub to figure out what was deployed and why. On platforms like Azure Dev Ops and Service Now, I could even define my own graph node or edge types, so, for example, I could directly express concepts or events such as that a certain commit, deployed in a specific release, broke a previously implemented feature, which was supposedly covered by a specific unit test that passed in the release that in fact broke the feature.

And now this is interesting because what we have is, in fact, not absurd. The costs may outweigh the realizable benefits in any particular context, but that’s not the same thing as saying the proposal itself is absurd. The native graphs provided by these services really are powerful because when used appropriately they almost automatically answer many of my questions of both chronology and relationship that must be collated by hand when stored in disparate systems. So it appears there may be a reason that these platforms are successful, even when some of their individual components aren’t as strong as more focused competitors.

Process Drag
One of our founder's catchphrases is “Why isn’t it done yet?” Even setting aside human inertia, the barriers to process change are many, and together I think of them as a sort of drag. At best it’s leaning into a stiff wind. At worst it’s like slogging through waist-deep molasses.

In particular, switching costs are major a barrier to experimentation and evolution in economies of any type, and no less in business software. A Git repo can be stored anywhere, but the substance of a pull request (its comments, timeline, and reviews) exists only in a proprietary form on GitHub, GitLab, BitBucket, etc. All the work management systems have roughly equivalent concepts for stories, bugs, and other work items, but you can’t just copy your complete work history from one system to another.

Iteration and compromise
It’s hard to even know what any given person means by “agile,” and in fact, many people I’ve asked don’t seem to quite know themselves. But one of the practical methods it often implies is iterating quickly by compromising perfection and completeness for the sake of incrementally delivering something that actually works. My first iteration, then, was to introduce two fairly simple incremental practices:

1. Git tag with semantic version: Before a commit is deployed to production, mark it with a native Git tag whose value is the faithfully semantic version of the application or component. We already use Git for source control, so this is just using an existing feature of a tool with which we are already familiar.
2. GitHub release with hyperlinks and summary: After deploying, also created a GitHub release attached to the Git tag, which includes the following information (which must currently be gathered manually and composed in markdown):

• The timestamp of the deployment
• A concise description of the functional changes included in the deployment
• A GitHub link to show the diff between this release and the previous one (by comparing the two associated tags)
• A link to the Travis build showing that this exact commit passed CI tests
• A link to the Heroku build where this exact commit was pushed

Conceptually, this produces the following graph:

The end product itself exists as a timeline in the GitHub releases UI, succinctly and visually summarizing change over time. Although the graph does not exist in visual form outside my illustration above, its edges can now be traversed through a mixture of automated or manually constructed hyperlinks. Most importantly, adding the GitHub releases with the format described above introduces an origin from which all the other nodes can be reached in just one or two moves, whereas before many things linked to the commit but not the other way around. In particular, it was difficult to make the connection all the way from the verbal discussion of a change in stories and pull request comments to an actual deployment.

Here’s a snippet of the new release history, which I went back and reconstructed from the last major release we did about 4 months ago:

Conclusion
We implicitly know that tools and processes can be useful. The mistake that is so often made is starting with a process or tool rather than starting with a goal. The contemporary business climate, with its fetish of all things “agile” and “startup”, all the more seems to pressure people to “disrupt” first and rationalize later.

Critical thinking cannot be boiled down to a series of rote steps, but having a few recipes at hand can still be a useful aid. I feel almost silly writing the following because it seems so damn obvious. To the contrary, throughout my career in business and software building, I have consistently observed that people usually do not follow such steps. So, for posterity, here is a basic recipe for making things better:

How to make things better

Think
1. Identify and describe your goals
2. Identify and describe the existing processes and tools that affect these goals, whether positively or negatively
3. Consider what process changes may assist in achieving those goals, taking into account the costs of and resistance to change
4. Lastly, consider if there are any tools that may facilitate the desired process, taking into account the actual goals of the process, the overhead of adding tools, and the limits on flexibility that tools can impose

Do
1. As much as possible, break the proposed changes into small pieces.
2. For each piece (but possibly in parallel)

• Identify what should be different after implementing the change
• Implement the change
• Compare what is actually different (if anything) to what was expected or hoped to be different

You will find that all of these steps require serious thinking. The good news is we are humans, and actually thinking is exactly what humans are ̶g̶o̶o̶d̶ ̶a̶t̶ uniquely capable of. So go be human!

~ Joshua, Software Developer