To infinity and beyond
I was working on the use of printed electronics with paper (think of digital interactivity within a normal book) when I got that phone call from a friend about “this spaceman who really wants to meet you”. I was curious, so I went along to meet someone called Mark at Canonical. I didn’t know what to expect. The first few minutes were certainly interesting.
Shuttleworth : “I’m Mark. I’ve been told you’re a good UX designer.”
Me : “I don’t know anything about design.”
It was an awkward pause. Then Mark realising the next hour was probably a waste of his time asked me to tell him something I did know about. I talked about evolution, the changes in the industry and before long we were into graphs, maps and cloud computing. The time flew by. We kept talking. I was introduced to others and in what seemed like lightning speed, I was working at Canonical. I had one job, to bring Ubuntu into the cloud. I called my friend, asked him what had happened. Steve just responded “I knew you’d get along”. Life is full of pleasant trouble makers like Steve.
The first day I arrived for work, I was all excited and had the usual confused look of a rabbit staring at headlamps. My boss, who also happened to be another Steve, did the usual rounds of introductions. That was an interesting moment. Whilst I delighted in the warmth of the people I met, the first five responses to my role of bringing Ubuntu into the cloud were negative — “it’s a fad”, “why are we doing that” etc. I knew I was going to have to build a cabal pretty quickly and create some momentum. However my first official task was to look at the virtualisation strategy that had been written. It was one of those “oh, what have I done” moments. Fortunately it didn’t take long to find others with common interests — Rick Clark, Soren Hansen, Nick Barcet and many others. Steve George (my boss) was also one of the most supportive people I’ve worked for, a good friend and then there was Mark. Without Mark none of this would have happened.
The problem to begin with was Canonical was focused on the server and desktop market. It was up against huge giants such as RedHat and Microsoft. It was making valiant, almost heroic efforts but Canonical was small. Many wanted to focus on the server OS, to generate revenue from support licenses and to a few then the Cloud was a distraction. The problem was one of focus and what I needed to do was change the mindset. To explain this issue and why it mattered I’m going to cover a number of concepts from the Three Horizons to Porter before returning back to Canonical.
The Three Horizons
The three horizons was a model put forward in the Alchemy of Growth, 1999. It discussed three views that any corporation had to take.
Horizon 1 : the core business which provides the greatest profits and cash flows that need to be extended and defended.
Horizon 2 : are the emerging opportunities and businesses that will drive medium term growth. These may include new ventures that you are investing in which are expected to generate substantial future profits.
Horizon 3 : These are ventures that should ensure the company’s long term future. They can include research projects or pilot programs or even investment in startups.
For Canonical, horizon one was the core support revenue. Horizon two included new concepts such as online storage, the app store and extending onto more devices. Horizon three … well, I’m quite convinced a few would have thought that included myself. Whilst this model of three horizons is a reasonable way of examining a company, I personally find it inadequate. I often find that some confuse it with the pioneer — settler — town planner model of organisation (chapter 4) by associating town planners with horizon one and pioneers with horizon three. To explain the weakness with the model, I’m going to use the map of mapping that I introduced earlier in chapter 7. To save you scrambling back through past chapters, I’ve provided that map here in figure 213.
Figure 213 — The Map of Mapping.
Let us now assume that we decide to use the map of mapping to build a new business. I’m going to take a part of the above map and concentrate around the provision of forecasting (i.e. anticipation of known changes) to others. I could have quite easily built a comfortable life around the weak signals that I had developed for forecasting change by creating a small boutique consultancy providing market and technological forecasts. The premise behind such a business is provided in figure 214. My purpose with such a business would have been to simply survive (i.e. make money), the user would be after an advantage over competitors and would likely measure this by the return on capital invested in a space. The business itself would provide anticipation services based upon known climatic (economic) patterns that use maps of the industry.
Figure 214 — Forecasting Service
Horizon one would be that boutique consultancy business. I’d have been protecting (i.e. not making creative commons) the twenty odd common economic patterns that I know about which impact the environment. I’d probably use a worth based mechanism (or outcome based as it is called today) for charging. I could also extend this map to cover in more detail the social capital components of trust and the activities needed to either perform the analysis or run the company. Remember you can map all forms of capital whether data, practice, activity, knowledge or social. Let us hypothesise that I had decide to build this company and by hook or by crook turned it into a small success. What would my horizon two be?
In this case, the diffusion of knowledge and evolution caused by supply and demand competition would drive many of those components to a more industrialised space. At some point, I’d have to prepare myself for my boutique consultancy entering a world where products did the same thing. I would know in advance that we’d have inertia to that, any shift from one stage of evolution to another (e.g. custom to product) causes inertia through past success. It’s one of the those climatic patterns. I’ve mapped this change in figure 215.
Figure 215 — Horizon two
But, with foresight — and I’d hope that I’d be using mapping on myself — then it would be relatively trivial to anticipate and overcome the inertia. How about horizon three? In this case, we get a divergence. I could for example focus on further industrialisation to a more utility service exposed through some form of API — Anticipation as a Service or AaaS for short. Of course, such as change along with mirth over the acronym would come with significant inertia created by any existing product based business model. Alternatively, I could expand into something new such as the use of doctrine for competitor analysis or the arms sale of context specific gameplay or even some novel, uncharted, higher order system that I haven’t even considered. I’ve shown these divergent horizon threes in figure 216.
Figure 216 — Horizon three
Now let us add the pioneer — settler — town planner model onto the horizon three map (see figure 217). Remember each team has different attitudes, which is what pioneer, settlers and town planners represent. Each team not only builds but operates and maintains their own work until such time that another team takes it away from them. The important thing to note is that horizon three consists of town planners or settlers or pioneers or all of them depending upon where I choose to focus.
Figure 217 — PST added to horizon three.
The horizons are context specific. You cannot simply overlay them onto a PST model or even the concept of evolution by saying “genesis is horizon three” as it depends upon where you are and the landscape surrounding you. Another thing to note is that the horizons can often be broadly anticipatable. This is the thing I find inadequate with the horizon model because without a map and the learning of common economic (aka climatic) patterns then it becomes all too easy to miss the obvious. It is why I find the three horizons useful as a high level concept but overall weak in practice on its own. It also fails to help me adequately deal with inertia or legacy.
The issue of legacy
In chapter 9, we examined the climatic patterns of co-evolution i.e. practices can co-evolve with the evolution of an activity. There is usually some form of inertia to a changing activity and this can be compounded by co-evolution of practice. In figure 218, I’ve taken the original diagram from chapter and added some inertia barriers for the shift from product to utility for both compute and also platform.
Figure 218 — Change of Compute and Platform
As previously discussed, there are many forms that inertia can take. However, the question I want us to consider is what represents legacy in this map? The two obvious areas for legacy are those trapped behind inertia barriers e.g. compute as a product and platform as a product (i.e. platform stacks). The next obvious includes those related practices i.e. best architectural practice associated with compute as a product. What is not so obvious to begin with is the issue that as components evolve enabling higher order systems to appear then the lower order systems become less visible and for most of us legacy. The departments that ran switchboards in most companies were once a highly important and often visible aspect of communication. For many companies, that activity has been consumed into either reception or call centres in much the same way that email has consumed the postal room. We still send letters to each other (more than ever before) but they are digital. In the map above, the role of the components underneath the platform layer are going to become less visible. Dealing with and managing infrastructure will become as legacy to most companies as the switchboard is today.
Hence another area of legacy would be the practices and activities below the platform layer which includes concepts such as DevOps. In 2017, such a statement tends to receive a strong negative reaction. Most react with the same forms of inertia as those who reacted against cloud in 2006. Many will claim DevOps is more than infrastructure as it’s about development and culture. Depending upon how far in the future you’re reading this from, you’ll probably be quite surprised by this and even more likely you will have never heard of DevOps.
As with all such things, DevOps was a child and reaction against the prevailing methods of management. It co-opted concepts from earlier schools of thought (e.g. ITIL) including iterative approaches, use of components, configuration management, services approach, a focus on users and measurement whilst simultaneously distancing itself from them. It added its own dogma and created a separate tribe. The same will happen in platform, a new school of thought will emerge that will copy and build upon DevOps but deny it has any relationship to it. DevOps will become “what my mum and dad does” as the rebellious child declares its independence and denies any inheritance from the former. Many of the genes of DevOps will be found in this new generation (though they will rarely admit it, painting DevOps as some form of strawman version of itself), some of the genes will become recessive and new genes will dominate.
I’ve marked on these main areas of legacy onto our map in figure 219. To do this, I’ve used the concepts of inertia and how industrialised components enable not only higher order systems but become less visible themselves. I’ve also added on a typical PST structure. As we can see, many of the legacy areas exist within the settlers and the town planning teams.
Figure 219 — adding legacy (a consumer perspective)
There is also a perspective to be considered here. I’m looking from the point of view of someone who consumes compute. If I’m a major provider, whether platform in the future or utility compute today then much of this is definitely not legacy any more than power generation systems are to electricity providers. From the perspective of a major provider then legacy would look more like figure 220 i.e. it will consist of activities (and related practices) that are stuck behind inertia barriers but not the impact of lower order systems becoming less visible. What becomes increasingly invisible to others (i.e. consumers) is still very visible to providers.
Figure 220 — legacy from a provider perspective.
Despite the unfortunate tendency of people to associate the town planning groups with legacy, it should be clear from the above that this is not the case. Cloud computing was has been all about industrialisation by town planners to utility services. The recent legacy has been past product models, a realm of settlers. If we take the consumer perspective from figure 219, then the future is a mix of settlers building applications, pioneers discovering emerging practices that combine finance with development (whilst denying any inheritance from DevOps) and town planners busily create the empires of scale around platform utility services. I’ve shown this future in figure 221 and it’s where companies should be investing in 2017.
Figure 221 — the future, from a consumer perspective
It’s important to note that legacy can be anywhere. It can be caused by a custom built activity which has failed to evolve or a product based business in a utility world. Legacy is simply a consequence of a failure to evolve and it is not associated with one group such as pioneers, settlers or town planners but instead all. When it comes to managing legacy then it’s really important to understand those points of change and the impact of co-evolution. This will become second nature to you but it’s worth practicing. There’s another perspective beyond the three horizons, beyond inertia and legacy that we also need to discuss. It’s the perspective of Porter’s forces.
For those unfamiliar with Porter’s five forces, these are rivalry within the industry, threats of new entrants, threats of substitution and the bargaining power of suppliers vs consumers. In this section we’re going to examine these five forces through the lens of the peace, war and wonder cycle (see chapter 9).
In the time of wonder, it is a battle to become established. The field is not yet developed and there are no “new entrants” as there are no established figures to be “new entrants” against. Everything is new, uncertain and uncharted. It is the wild west, ‘ere be dragons and the home of split infinitives. The consumers hold the power and it is they who decide whether this industry will succeed or not despite their initial inability to know whether they need it.
In the time of peace, there is a constant tug of war between supplier and consumer power over the products produced. The developing giants are normally well protected from new entrants in a game of relative competition. The exception is the occasional threat of substitution. It is this substitution by a different product which tends to be the dominant factor.
In the time of war, new entrants providing a more industrialised form of the act threaten the existing giants that are stuck behind inertia barriers. It becomes a fight for survival for these giants and they are often poorly equipped. It is not a case of a product becoming substituted by another product but instead an entire industry being changed to more industrialised forms. It is often assumed that the shift towards utility provision means centralisation but this is not the case.
Whilst the interaction of all consumers (demand competition) and all suppliers (supply competition) drives the process of evolution, the question of whether a specific activity or data set centralises or decentralises depends upon the actions of individual actors (suppliers and consumers) in this market. For example, it would have been relatively trivial for the hardware manufacturers to create Amazon clones and a price war in the IaaS space around 2008–2010 in order to fragment the market by increasing demand beyond the capability of Amazon to supply due to the constraint of building data centres. I had these exact conversations with Dell, IBM and HP throughout 2008 and 2009. I even told them their own inertia would fight against this necessary change and they would deny the existence of the punctuated equilibrium until it was too late. The fact they didn’t act and lost their own industry is entirely the fault of their own executives and also one of the major factors why have seen centralisation in the IaaS space.
Centralisation depends upon the actions of specific actors (in this case the inaction of hardware suppliers and hosting companies). In the future, this may in fact yo-yo from centralised to decentralised or find a balance between the two (as with electricity provision and self generation). Such a change in the means of production is however unlikely to change the interfaces themselves i.e. a shift from central to self-generation does not mean a change in voltage or frequency for domestic power provision. The future interfaces of computing have already been defined.
The point to remember with Porter’s forces is the balance between these forces tends to change as any component evolves. It also isn’t static within a stage of evolution — for example the yo-yo between centralisation and decentralisation with a corresponding yo-yo between Supplier and Consumer bargaining power. However as a general guide, I’ve provided in figure 222 the most dominant forces you’re likely to encounter.
Figure 222 — Porter’s forces and evolution
With a basic understanding of horizons, Porter’s forces and legacy then we can now examine the business of Canonical. The horizon one (core business) was related to selling support on the server OS (operating system). However, compute was evolving to more utility provision. Hence, with the exception of large cloud providers then the server OS support was likely to become a legacy business. Instead, we needed to focus on horizon two and the commercial use of guest OS on top of these large virtualised computing environments. We understood that companies would have inertia to these changes and being a shift from product to commodity forms it was likely to be a punctuated equilibrium (period of rapid change). We also understood that the biggest threats into this space would be new entrants and given the state of strategic play in many companies then we were likely to see centralisation. I’ve drawn these concepts onto the map in figure 223.
Figure 223 — the changing market
We also understood that co-evolved practices would emerge, that we were unlikely to see significant savings in IT but instead increased development activity and that a further horizon, the shift of platform from product to utility was possible. I’ve marked up these horizons onto figure 224.
Figure 224 — the horizons.
In terms of play, we understood that moving fast and land grabbing the guest OS territory was essential. To help in this, we also needed to support those developing applications or building tooling around those co-evolved practices. If we found examples of platforms plays in this space we also needed to be invested in this. We understood that many potential customers would have inertia hence we’d have to provide some forms of transitional or private cloud offer even if this did nothing more than get the conversation started.
We also knew our competitors had inertia. As soon as I discovered Red Hat salespeople were rewarded bonuses based upon satellite subscriptions (used for security updates) then I quickly set about promoting a message that security should be “free” in the cloud. There’s nothing like threatening someone’s bonus to get them to turn against a change. Our focus was clear within my cabal. Mark did an amazing job of turning this into the entire company focus. Rick and others set about putting in engineering effort to make it happen. Steve gave me all the firepower and cover I needed. For my part, I mainly focused on promoting Ubuntu’s cloud message, being involved in the community, highlighting targets to bring on board and trying to stop people rebuilding or getting in the way of things that the community was doing.
An outline of the play is provided in figure 225 and the result in figure 226. Within eighteen months, Ubuntu went from a small part of the operating system to dominating the cloud guest OS. My part was a minor but instrumental role and I have to applaud the marvellous teams at Canonical and within the community for making it happen. A small company of three hundred took on the might of two giant hordes but unlike the Spartans, this time we won. My proudest moment came from hearing a CIO talk about how “the future was all RedHat and then suddenly it was all Ubuntu”. I played a small part in that.
Figure 225 — our focus
Figure 226 — the results
I often hear people talk about how Canonical was lucky, well there’s always some element of luck but the moves were deliberate. Obviously, people can just say the timing was lucky but they’d be wrong on that as well. I had a helping hand with timing thanks to Gartner. They probably don’t even realise but I think it’s worth explaining.
On the question of timing
I’m not a big fan of Gartner but figure 227 is one of the most useful graphs they’ve ever produced. It’s a hype cycle of emerging technologies created in 2008. It uses the earlier y-axis of visibility which later on became expectations. How can the axis change whilst the graph remain the same? Ah, that’s the beauty of it but first, a bit more background.
Figure 227 — Gartner emerging technologies, 2008
During my time in the wilderness prior to Canonical, I had been looking at various ways of measuring impacts from evolution. One of the issues I had come up against was the evolution of any single act creates two waves of opportunity. One of these waves is focused on differential value (i.e. it’s something you have but I don’t) and the second wave is around operational value (i.e. we both provide this but you do so more efficiently). Both the waves appear to have a learning element and then a sharp decline as the change diffuses and evolves further. I’ve provided examples of these waves in figure 228.
Figure 228 — An example of different waves of value.
Of course, opportunity is only part of the equation. There are volume effects and the cost involved particularly in development of something novel. There’s also risk as the uncharted space is by its very nature is uncertain. However, I developed a generalised benefit curve which for differential value is shown in figure 229. An almost identical benefit curve appears to exist for operational value but that occurs much later in evolution and is related to the co-evolved practices that emerge.
Figure 229 — A benefit curve for differential value
From the benefit curve, the early stages of genesis are all about investment. As it evolves, the cost of production reduces and we start to realise some of the benefit. We’re still in the custom build stage, others are starting to copy but in general the cost of production is reducing fast enough to overcome any differential loss due to copying. Alas, at some point the cost of production is low enough and the activity defined enough that someone produces a product. On the upside the cost to implement is plummeting but alas, the differential value is declining faster as more companies actually implement. The models I developed all had variations of this shape. I’m not comfortable enough with the data, so think of it more as a mental model and a possible curiosity.
Whilst exploring this space, I then became fascinated by timing issues. Let us pretend we’ve recently read a whitepaper on some marvellous new activity. That activity is described as having some benefit but it also involves cost. By the time I get around to implementing the activity then it will probably have evolved. It might provide a different benefit to what I was expecting i.e. it costs less because it’s a product but there’s little differential value as everyone else is also doing this. I’ve superimposed the evolution of an act onto the benefit curve in figure 230 to highlight this point.
Figure 230 — Changing benefit with evolution and implementation
I then modelled this delta between what I was expecting to get and what I got over time. The model I used made lots of horrible assumptions, it’s uncomfortably close to voodoo and is about as solid as a tower of jelly. At some point in the future, I might go and revisit this but I don’t normally mention this little side journey. However, there was one remarkable thing about the delta expectation curve over time — it resembles a Gartner hype cycle — see figure 231.
Figure 231 — delta expectation over time (the expectation curve).
We have the same peak of inflated expectation and the same trough of delusion. My first reaction was horror.
The evolution curve on which mapping is built uses ubiquity versus certainty. If I can model from Gartner’s hype cycle to evolution then I can take the points on a hype cycle and measure precisely where something is on the certainty axis of evolution. For things that are uncertain then this should be impossible as the ability to precisely measure something which is uncertain is the stuff of magic folk. My first reaction was Gartner’s hype cycle proved evolution was wrong. I was a bit perplexed at that point especially since I had found mapping so useful. Fortunately, I met with a friend who pointed to a great big hole in my argument. I was assuming that Gartner’s hype cycle was based upon the measurement of some physical property. If it wasn’t, if it was just aggregated opinion (of consultants, analysts or industry) then there’s no measurement of the uncertain as it’s just opinion. It’s an opinion of where something is, not a measurement of where it actually is. As I subsequently found out, the hype cycle is subjective opinion.
Along with being quietly relieved that I hadn’t yet disproved what I was finding useful, it also opened up a new opportunity. I have two benefit curves — one for differential value and one for operational value. They both shared a common expectation versus time pattern. If I look at an evolving component then where it appears in the early stages on the expectation curve for differential value can be the same place it appears on the expectation curve for operational value when it’s more evolved. See figure 232
Figure 232 — Evolution of an act on differential and operational expectation curves.
I also had a weak signal using publication types that could identify when things are likely to start to industrialise and enter a war (see chapter 9). I’ve reprinted the last analysis on this that I undertook in 2014 in figure 233. What I’d like you to notice is that the shift from product to utility for computing infrastructure was well into a war in 2014. Whereas the war for 3d printing and the use of commoditised 3d printers is some way off.
Figure 233 — When is the war likely
In 2008, I already knew (from my weak signals) that we were entering the war phase for computing infrastructure whereas 3d printing had a long time to go before it started to industrialise. I also suspected that both a relatively novel activity (e.g. 3d printing) and an industrialising activity (cloud) could appear at the same place on two different expectation curves — one for differential value and one for operational value (figure 232 above). So, let us look at that Gartner hype cycle again and highlight two components — cloud computing and 3d printing.
Figure 234 — Cloud computing and 3D printing.
They both appeared at roughly the same place. This told me something which I’ve subsequently found quite useful. The Gartner hype cycle doesn’t distinguish between differential and operational value as both are on the same curve. So, why does that matter? Well, in the case of cloud computing, which was the industrialisation of computing and all about operational value then you’d want to be going “all in” during 2008. Being in the early stage of this expectation curve just reinforces the point that people are learning about a change which you absolutely want to be a first mover to. The last thing you’d want to do is wait until it reach the plateau of productivity by which time the war would be well and truly over. If you’re a vendor, this would be curtains. Gartner even calls out that this is moving fast with its time to mainstream adoption for cloud (light blue circle).
However, in the case of 3D printing then you do want to wait or be a fast follower. It has a long long way to go before it industrialises and you’ve got an entire product stage it has to evolve through. In fact 3D printing will reach the plateau of productivity and see relatively widespread adoption as a product long before it industrialises. At some future time (2025–2030), as it starts to industrialise then it’ll probably reappear in the technology trigger usually under a slightly different meme. When it comes to 3D printing then you could wait a bit and get involved in the product space or wait much longer until the “war” is upon that industry at which point you’d need to go “all in”.
Two points — cloud computing and 3D printing — on almost exactly the same position of the hype cycle required radically different approaches to investment and strategy. One was “all in”, the other was “wait and see”. Being aggregated opinion, I do find the hype cycle quite useful as long as I separate out what stage of evolution something is in first. I often talk to CIOs who tell me they invest when something is in the stage of enlightenment. That’s a fairly reasonable way of losing every major technological war in business.
For me in 2008, this hype cycle helped reinforce the message that we had to go all in, it was a land grab for this territory. I also took comfort that many of my competitors probably read exactly the same hype cycle and thought they had time. Let us emphasise that point, I was going “all in” when competitors thought they had time — it’s a help yourself to the future buffet with no-one saying you can’t have 7th helpings because everyone else got the date wrong. Thank you Gartner, you probably have no idea how much you’ve helped me. Better luck next time IBM, HP, Dell, RedHat … assuming they survive what is to come.
Anyway, the gameplay above was 2008 to early 2010. In mid 2010, after capturing pretty much the entire market (a space that has massively grown since with Ubuntu still the “top dog” in 2016), I then headed back into research. My work was done. Naturally, I left Mark and others with a variety of plays to use along with a specific focus on the platform space. I don’t necessarily agree with all the steps they’ve made but I respect their choices and they play a good game. I suppose, that’s the real point — they are playing the game not me but in some small way I helped them to improve. Before we dive into the research space, we should take a peek at another part of my journey into Government. Hence let us boldly go into the next chapter.
The book so far
Chapter 1 — On being lost
Chapter 2 — Finding a path
Chapter 3 — Exploring the map
Chapter 4 — Doctrine
Chapter 5 — The play and a decision to act
Chapter 6 — Getting started yourself
Chapter 7 — Finding a new purpose
Chapter 8 — Keeping the wolves at bay
Chapter 9 — Charting the future
Chapter 10 — I wasn’t expecting that!
Chapter 11 — A smorgasbord of the slightly useful
Chapter 12 — The scenario
Chapter 13 — Something wicked this way comes
Chapter 14 — To thine own self be true
Chapter 15 — On the practice of scenario planning
Chapter 16 — Super Looper
Chapter 17 — To infinity and beyond
Chapter 18 — Better for Less