Patterns, GitHub, and Misplaced Expectations
Kyle Brown
I’ve written before about how I’ve been involved with writing and using Software Patterns for well over twenty-five years now. But even with that amount of experience, one of the things that keeps me excited about the field is that I found that I can always learn new things and new approaches by reading patterns written by others and by writing my own.
One of the more unusual projects I’ve been involved with has also been one of the longest patterns projects that I’ve been involved with. It all started back in early 2016 when several things came together all at once. The first is that at work, I found myself in the middle of a huge cloud transformation project with one of our major customers. This project was great for a lot of reasons, but one of them being that it stretched me in a bunch of different directions — not only was I expected to help the client define a brand new cloud-native architecture for their customer-facing applications (which is what I normally do in my day-to-day work) but I also found myself advising them in how to scale an Agile transformation with XP, apply Lean Software Engineering approaches in their teams, and helping them to adopt Site Reliability Engineering, all at the same time.
One of the principles I’ve always followed at work is that if I’m ever asked a question three times, I write it up as a blog, an article or a paper, because if three different people I meet need to know the answer, it’s probably generally interesting to the industry at large. This particular project had me answering questions like this on a daily basis, usually several times a day. As often happens when I find myself in that situation where I’m answering multiple questions — my first thought was that I should write a pattern language about this.
In particular, I kept finding myself telling folks at the customer about some of the basics of Microservices design, and about refactoring applications to use Microservices as part of a cloud-native transformation. Now, at this time, Microservices were still new — James Lewis and Martin Fowler’s paper had only come out in 2014, and Sam Newman’s book on the topic had literally just come out a few months before. What I was seeing was a tight connection between some of the advice in Sam Newman’s book about Microservices (particularly the use of BFF’s in multi-channel applications) some of the experiences that we had had in developing Single Page Applications that connect to small Java programs (which I had written about in our 2014 book Modern Web Development with IBM WebSphere) and in our experiences in applying Martin’s Strangler Fig Application pattern as part of a refactoring effort.
What this project was showing us was the need to expand out some of the ideas of Microservices to handle the problems that emerged from connection to back-end systems of various types. The original Microservices work was mostly concerned with greenfield applications, and it wasn’t until a couple of years later that the first book on Microservices refactoring would be published. Drawing from my experiences with refactoring to Microservices, I finally put all of my thoughts together into a pattern language we workshopped at the PLoP 2016 conference in Illinois.
But it was at that conference, after having discussed the paper that my first epiphany occurred. One of the other folks workshopping a paper at this conference was the incredible Cees de Groot, who at this point was working at PagerDuty, and who had a set of experiences that were weirdly intersecting with mine. While I was focused on the application development side of things, Cees had built up a ton of expertise on the infrastructure side of things — and we found that our two pattern languages really looked like two halves of the same coin. So during a break, we found a whiteboard, and quickly sketched out how our two pattern languages could intersect and come together.
Practically this meant things like realizing that VM Automation (from Cees’s language) is critical to ensuring the success of a Lift-and-Shift (from my language) to the Cloud. There were dozens of other connections that we made like that. But then we had another question — we felt we were on to something important here — how could we continue to work on this given we were separated by several time zones and different working schedules? Previously, I had faced similar kinds of constraints in writing books with geographically diverse author teams — so one of the things that I mentioned is that we had been successful in using a SCM system to store versions of documents. We also discussed the possibility of doing this in Google Docs (which I had just used for my last book project). It was at this point that Cees (possibly putting the two together) came up with the idea that at once was obvious and revolutionary.
“Let’s just do this in GitHub”
As we looked into this, the benefits became increasingly obvious and I became more excited about the idea. I had been using Git for at least a couple of years prior to this, but was thinking of it mostly as a code repository. We had published several projects as part of my work with the IBM Cloud Architecture Center that had relatively long README.md markdown files to explain the use and installation of the projects. However, I had never thought about the idea of having a GitHub project that ONLY consisted of Markdown files.
The cool thing about this is that as we thought more about this, we realized that GitHub and Markdown are extremely well suited to writing pattern languages. Let’s review a bit about what a pattern and a pattern language is in order to understand why this is true:
Patterns are often said to be “A solution to a problem in a context”. What this really means is that Patterns (an idea first conceived of by Christopher Alexander in The Timeless Way of Building and A Pattern Language) are a way of representing information as structured text that help readers understand how particular problems can be solved when they encounter them. When Alexander was writing his books, he was thinking about applying this to the problems of designing buildings, towns and communities, but starting in the late 1980’s, people in the software development community found the general idea was applicable to their domain as well.
One of the key ideas about Patterns is that they may sometimes be standalone, but are usually more useful when combined together. Alexander made this point explicitly several times. For instance, in A Timeless Way of Building Alexander wrote: “The patterns will only allow me to form a single coherent image in my mind, if the order that I take them in allows me to build an image of a design gradually, one pattern at a time.” This idea of patterns helping you form a coherent image built up bit by bit, pattern by pattern, is exactly the way that pattern languages are supposed to work. In the patterns community, this is sometimes referred to as generativity, in that if you read the patterns in the right order, they will help you generate new concepts — to form a new design in your mind by layering idea on idea.
In the software world, this jibes with the similar kinds of discoveries we have been making about the Software design process over the years — going all the way back to Barry Boehm’s “Spiral Model” of software development that proposed incremental development back in 1986. The Agile world, starting with XP, has encouraged this idea of building up applications piece by piece, particularly with the maxim “You aren’t going to need it” — which means that planning too far ahead and trying to work out all of the details of a design will usually end in tears, so you should only build the pieces you know about, and only when you need them. This assumes you can always add on new ideas later as you understand the problem better.
But practically, what this means is that you have to allow for multiple different paths through a pattern language. You can’t predict, a-priori, what problem your reader is actually facing — everyone is starting from a different place and likewise applying the pattern language to different problems. In Patterns terms, that means that patterns have different contexts — different starting points. You have to plan for navigating these different paths — and the best way to do that, as Ward Cunningham first discovered back in the late 80’s when he first introduced Patterns into the software development community, is through hyperlinks. A critical experience with doing that was when he implemented a hypercard stack to represent a set of patterns — this influenced his thinking process and led him (in 1994) to invent Wikis, which were originally meant for patterns authoring and sharing.
The critical lesson here is that you have to not only have a way of quickly authoring text and including graphics, but you need an easy way to create the hyperlinks between the patterns. A good pattern language should be heavily linked — allowing for a reader to navigate it in nearly any order they want. In a book, that’s sometimes a little challenging — the best idea anyone ever thought of was the one that the book Design Patterns employed where they always referred to patterns in the book by both name and page number (for instance Command (233)). However, on the web, using a hyperlink is the obvious solution.
So going back to Github, it had the following advantages:
(1) It’s easy to include text (in markdown files) with simple formatting commands that cover 99% of the kinds of formatting you need.
(2) It’s easy to include graphics in the form of graphics files that can be rendered and displayed in line with the text
(3) It’s super-easy to include relative links to other pages in your Git repository.
(4) It’s a source code repository! Nothing you do is ever lost, and you can reconstruct earlier versions of the pages easily. You can also restrict who can make changes to the pages, while allowing others to make suggestions (Issues) or even propose contributions (Pull Requests).
(5) It’s FREE!
After this second epiphany, we quickly copy-cut-and-pasted the patterns from our papers into a GitHub repository, patched up links between them, and sat down and admired our handiwork. We thought — great! We’re on our way to a book now! And then it sat unchanged for nearly two years.
Why? Well, life, in a word. We had two deaths in our family in quick succession that necessitated my wife and I having to get her parent’s house ready for sale, followed by major changes in my work and yet another book project (The Cloud Adoption Playbook) intervening.
The story picks up again in 2018. By the summer of 2018, I started working with Chris Hay on a set of patterns about Container DevOps that looked like they would tie nicely into the patterns Cees and I had already written. By the time that paper wrapped up in late 2018 we had a plan to all meet at PLoP and get started on the book project again. This time, Cees was the one to push things off, so that moved things back a few months. Then I had a change at work, becoming a manager again for only the second time in my 30+ year career, and also started building and moving into a new house. By the time January of 2020 rolled around, I had a couple of additional IBMer’s lined up to help out with the project, and it looked like it would fire up again…only for 2020 to happen.
Early 2020 was kind of nice for the site — when we were all getting used to working remotely, but before the ritual of multiple zoom calls a day began, I was able to expand the site yet again to add a set of patterns about Event Driven Systems that I had co-written way back in 2006 for an internal IBM Academy of Technology conference. We were working on new Event Driven Systems with a couple of customers at the time and these patterns were applicable to the architectures we were building, so I resurrected them, cleaned them up added them to the site, and brought them up to the latest technologies. But by Summer 2020, we were in full remote working swing, so the GitHub was set aside yet again.
By early 2021 the patterns had been sitting on GitHub for five years, accessible only to the authors and a couple of additional people who had specifically requested access. Then, in March, one of my friends with the University of Porto in Portugal asked if it could be made publicly accessible for some of his students to use. At this point, a couple of things had happened.
(1) I was pretty sure that I wasn’t going to find time to work on this more than a few minutes a day, which is fine for a web site, but not enough for a book project. Not that I won’t ever turn this into a book, but I knew it wasn’t happening immediately.
(2) I looked back on a couple of projects we had been doing at work and had a third epiphany — the site was valuable not just for the patterns, but as an example of the process we used to write and publish the patterns!
You see, the positive experience I had with writing the patterns for cloud adoption on GitHub had made me into a bit of an evangelist for writing architectural guidance in GitHub. I wrote a well-received Medium article on the idea, had presented at an IBM internal conference on it and also at the IBM THINK conference on the subject, and had led the execution of the approach in two major projects within IBM. During those last two projects, we had moved from just plain GitHub and Markdown to publishing the results in GitPages — which had become remarkably easier to use since it was now as simple as setting up a configuration option in your project settings.
So, after one frantic weekend of working to fork the original repository (which Cees owned), clean things up, get everyone’s permission and work through some Jekyll issues, I made the site available to everyone at http://cloudadoptionpatterns.org on Monday March 8, 2021.
So, if you are looking for guidance on how to get started with adopting the cloud, and need an architectural viewpoint on that, please visit the site above. If you are just interested in how the patterns idea can apply to modern technologies — then also visit, but take a step back and look at how the patterns are organized and interrelated.
But the really cool thing is that if you have feedback on how something could change, then simply open a GitHub issue on the site! In fact, I might even consider taking potential collaboration into account through a pull request — we’ll see how that works out. In the meantime, I’ll spend a few minutes a week on the site tweaking the patterns, expanding the number of diagrams (I want to eventually get to one diagram per pattern, as Alexander had in his book) and adding new examples of technologies and solutions that implement the patterns.
Maybe one day we will all find the time to publish a book from this, but until then, we believe the GitPages site itself is extremely valuable, and hope you can all get something useful from it.
