The sociotechnical bloom: upgrading business to an open operating system
After writing “Business, management and strategy in the new paradigm”, I realized I’m really talking about a “sociotechnical operating system”, one founded on the principle of openness. Where the “old OS” is about carefully controlling the environment (internally via organizational silos, externally via competitive moats) to extract value, the “new OS” is all about giving up that (illusion of) control and embracing the positive externalities of exchange.
Of course, that view is heavily shaped by my experience in the software industry: like most people in that field (and many in adjacent ones), I have been a beneficiary of the open source cornucopia throughout my professional and academic life, and so I often fall into the trap of thinking that the overwhelming advantages of openness — not just in software but in all the technologies and mechanisms that make up our economy and society — should be too obvious to need stating.
Of course, quite the opposite is true: Even as software continues eating the world, most people outside the industry are wholly unaware that this trend is fundamentally driven by the self-sustaining momentum of open source. And worse, even many people in the software business hold inaccurate notions of why open source is successful. So it’s no wonder that the mental leap from open source to open everything has had a hard time landing, even though it’s been decades in the making.
In this article, I will argue that “open everything” will create truly gigantic opportunities for value creation, particularly as we consider the challenges — greening our economy, weaving a new global social fabric — that lie at the core of the ESG movement. Indeed, I am convinced that our best hope to deliver solutions to those macro challenges is to embrace the principle of openness. Those few who “get it” will be uniquely well positioned to lead the way in creating value in this new paradigm. I will show how the future, “open by default” tech and business landscape will look like, and explain why, far from being far-fetched, it’s actually a straightforward consequence of simple economic principles at work.
But first, I’ll show how powerful open source software really is in practice and in theory. And it all starts with a story…
The 2010s: A typical software development journey
In 2010, I founded a startup with a friend. As often happens, our company had a “left brain” and a “right brain”: I was good at numbers and logic, he was a marketing and sales genius. We mutually agreed that he should be the CEO, and I — my experience in development at the time being limited to toy CS projects at university and simulation models in consulting— was nonetheless supposed to be the CTO and head of product. And so, as my team of two set out to design our amazing product (Brazil’s pioneering taxi app!), and “learn professional software engineering” on the fly, we followed the rulebook.
Warning: The next paragraph contains concentrated doses of software geek speak. Feel free to skip it if it makes you uncomfortable!
We started with the most popular stack for Web development at the time: the PHP programming language, the Apache HTTP server, the MySQL database, and the Linux operating system under it all. The Web version of our product used the JavaScript library jQuery to produce dynamic real-time map updates that looked like magic back then (at least to me!). Eventually, all the data traffic between drivers and passengers required us to re-architect our product many times: first adding in the RabbitMQ message queue to decouple networking and processing; then a full rewrite to the “new hotness”, the Node.js runtime — itself built on Google’s V8, a super fast JavaScript engine, whose coolest feature was an enormous repository of ready-made components (“packages”), called npm. We then blended in the more “advanced” databases MongoDB and PostgreSQL.
By 2014, our stack compared favorably to what bleeding-edge companies were doing — even though we were still barely scraping by financially and definitely didn’t have money to hire the hottest developers in the biz (or at times, any developers at all). Indeed, the tech was improving at a rate so fast that I had a hard time even keeping up with software updates and deciding what tools to adopt next. It allowed us to build a blazing-fast, scalable and reliable product with a tiny team of engineers.
And of course, the punchline: we had all of this tech for free, with no strings attached. All of it was remarkably easy to find, learn, use, deploy and maintain. We could look under the hood whenever we wanted — whether to understand how to use it more effectively, to make up for the occasional lack of manuals and documentation, or to debug an issue. However, it usually turned out to be us who were holding it wrong: because so many other app developers out there were using those same tools (tens of thousands in many cases), all bugs were quickly found by someone else, most were quickly fixed, and the patch was instantaneously made available for all of us to benefit.
Considering only the “top application layer”, our own medium-sized project was probably less than 30% made of our own code, the rest being open source packages and libraries. And that was just the tip of the iceberg: the rest of the stack, down to the low-level routines driving the hardware in our servers, was entirely the product of the open source community.
This is the unstoppable power of open at scale as we experienced it: blueprints and infrastructure for almost everything we could imagine needing, not just “for free”, but also continuously improving.
If I haven’t yet made it clear how magical this is, imagine if you could design, build and deploy an airplane at the cost of a couple of engineers working for a few months (plus materials). Let’s say we do a startup to take advantage of this miracle— let’s call it WeFly. Not only would our snazzy WeFly planes launch with all the latest features for safety, energy efficiency, speed and comfort built-in, but whenever someone else figured out a way to make it even better, all of our planes would also get instant updates — in-flight, if we wished so. We would create an initial design with all-standard components and maybe a few cosmetic additions: yellow and purple fluorescent interiors! After we got comfortable, we could try a radically new concept, by rolling our own custom components: parachute ejector seats, for passengers in a hurry to land!
All throughout, even as we’re selling WeFly airplanes — or using them to operate an Airplanes-as-a-Service business (VCs are pouring money into shared air rides these days!) — we can give back to the community by sharing the specs for our newfangled ejector seats. Sure, our competitors will benefit from the innovation, but let’s be honest: if that’s our competitive differential, WeFly is probably screwed already.
The miracle of spontaneous capital formation
Translating to economics speak: open source is a commons of intellectual capital (i.e., a shared resource made up of intangible assets such as knowledge, blueprints, processes, personal relationships). This commons grows in value both as contributors actively create and develop assets, and as users more widely adopt and depend on existing assets.
Now, commons are more famous for their tragedies: free-riders taking more out than they put in. One would imagine it would be even worse for open source, since it’s a totally open-access commons, with no visible admission cost. But in this case the exact opposite happens: since intellectual capital can be put to use without being depreciated, participants put more in on average than they take out. This holds true even if there are relatively few contributors and many non-contributing users (unless those are actively depleting the commons; we’ll discuss some ways this can happen below). This means the total value of the assets increases steadily; this in turn attracts more participants, including more contributors who believe in the future value of the commons, creating a positive feedback loop.
To most participants, it looks like magic indeed: free assets that grow indefinitely and pay dividends on demand. What’s the catch? Well, there are two:
- Critical mass. The commons only actually starts paying off after a substantial and sustained up-front investment, usually incurred either by dedicated individuals working in their free time, or by organizations taking a long-term ecosystem view (more on this below). This is usually described as “sponsorship”, but I consider that misleading; as we’ve seen, it’s a real investment with real and measurable returns, although with a somewhat unintuitive business model. The investor/contributor forsakes the chance to make money directly from licensing the product, but rather adds it to the common repository, where use and contributions from the rest of the community will further improve it (see below).
- Rules of the road. In order to grow and produce dividends, the commons depends on a functioning governance system — including the legal framework governing software licenses, committees and consortia who uphold technical standards, and the informal but crucial communities that serve as the “last mile” of governance. Without this “glue”, the commons can fall victim to code forks (contributors each create their own copy and “go their own way”, eliminating the benefits of larger collaboration), capture (a dominating contributor steers the codebase into being incompatible with competitors’ systems), etc.
Common myths about open source
As I mentioned in the introduction, there are many, including “software industry insiders”, who hold mistaken mental models about open source. As these misconceptions are relevant also to the other forms of knowledge commons we’ll discuss below, I’ll explain why they are inaccurate.
“Open source is a utopian socialist fantasy.” I think this objection stems from an outdated understanding of how money is made in the tech world. In the past, the vast majority of software revenues were indeed from licensing — this is how Bill Gates became the world’s richest man. However, as software “eats the world”, it has graduated from a separate segment into a crucial enabling infrastructure role for all other industries. That is, the vast majority of software is now used not as an end product itself, but as a component of added-value products: whether as “apps” using software to deliver consumer products or services (Amazon, Uber, AirBnB), “managed software services” providing technology to businesses (cloud providers like Google Cloud, AWS, Microsoft Azure; business SaaS like SAP; professional services like IBM or RedHat), or software powering hardware devices (smartphones, laptops, “Internet of Things” devices, etc). As such, most software development has much more value as part of a commons than as a potentially licensable product. (Incidentally, this is well demonstrated by Microsoft becoming the #1 open source contributor.)
“Open source is based on unpaid labor.” I couldn’t find authoritative data on this, but based on GitHub stats, the majority of open source development seems to be done by employees of aforementioned tech companies. Further, rather than using their free time to work on passion projects, these developers are working on officially company-sponsored open source tech. While there is certainly a long tail of unpaid contributors involved, I can’t find any large project that isn’t primarily driven by a tech company or a consortium of them.
“Open source only works for the software industry, where unit production costs are zero.” This, I think, also stems from an outdated, “static” conception of economics. In the contemporary, “dynamic” view (famously expressed by Schumpeter as “creative destruction”), what drives the dynamics of businesses and markets is not the today, but the tomorrow: the development of new assets and capabilities, which expand the value added by the company’s products. This is primarily a process of learning (knowledge creation), with the end result composed of software, physical assets, and the tacit intellectual knowledge of the company (processes, skills, structures) that generates production. Correspondingly, the relevant fact about open source is not its impact on production cost, but on the development process: by tapping into the open commons of freely reusable knowledge, companies can thus develop value streams much faster and cheaper.
“Open source is all or nothing.” Many of the founding fathers of open source did (and some still do) espouse a radical view, founded in political and moral arguments, to the effect that “information wants to be free”, meaning that all intellectual creation is supposed to be open. I won’t take a philosophical stance here, but my economic argument does not assume this strong form, merely the weaker form that there be some institutional protection to avoid the commons being depleted. There might be instances where keeping some code proprietary — whether temporarily or permanently — is the “right” answer, allowing developers to own some of their creations while continuing to give value to the commons by using and interfacing with open components.
Historically there has been a long period of “storming” in the community on this topic, leading to the emergence of alternative rules of the road (specifically licenses), some of which are more permissive than others regarding the use of open source from non-open projects. While the controversy is not fully resolved yet, the community has long since settled into a “happy medium” where permissive licenses like MIT and Apache allow open and closed source to interoperate, to the benefit of both.
Emerging pillars of innovation: Open research, open designs, open collaboration
The usefulness of an “open commons of freely reusable knowledge” is by no means limited to knowledge materialized as software. Although software did lead the way, we already see commons developing in all areas of productive knowledge, as the same economic pressures apply, and pioneers find ways to build critical mass and institute rules of the road. In particular, the following three crucial areas are showing signs of progress.
Scientific research. Although the academic mainstream has long practiced secrecy and insularity (and has strong institutional incentives to do so), younger researchers have broken with tradition and created communities (Blue Obelisk), institutions (Open Knowledge Foundation, Center for Open Science), and platforms (ArXiv, Figshare, OSF) to share data, methods and findings broadly. The benefits of open research have since become obvious, with success stories like ArXiv, now the “first place to read about new research” across many fields, and “massive citizen science” projects like FoldIt making the news.
Technical designs. While the open research movement could at least leverage the scientific establishment’s professed goal of advancing humanity’s knowledge, the pioneers of openness in the technology industries (other than software) have had it even worse, as the mentality in almost all tech companies has always been to aggressively protect their trade secrets and intellectual property. Even extremely successful projects like Raspberry Pi and Thingiverse still cater mostly to DIY hobbyists and educators. Ambitious efforts like Open Source Ecology are slowly growing, driven by volunteers. And even in the electric vehicle industry, which promised to be a pioneer of open innovation, with patent release pledges from Tesla and Toyota, the “opening” seems so far much more restrictive than what’s necessary to fully drive the growth of the commons.
Collaboration. While a commons of codified knowledge is already incredibly powerful, we can also expand the concept to tap into the vast majority of knowledge which is tacit, living in people’s heads — including all the new ideas that haven’t yet been materialized. We know that those ideas come to fruition by connecting knowledgeable and talented people into self-managed, highly motivated teams. This leads us to the notion of open collaboration between organizations, a notion highly antithetical to the traditional paradigm, for it challenges the very concept of the discrete corporation. However, the same economic benefits to openness apply here: ultimately, companies that find ways to connect with each other and collaborate fully will produce more value quicker at lower cost, building a self-reinforcing network of trusted partnerships which further accelerates the production of value via innovation.
This model of open collaboration is currently visible only in limited form, at cross-company technical forums and consortia developing open ecosystems such as Android’s OHA and the IoT Consortium. In contrast, what I’m describing here is that the company itself becomes the platform (or a hub in the ecosystem), as it offers its internal value streams to external collaboration, to the extent that the hard line between inside and outside becomes more of a fuzzy membrane. This notion is being piloted at companies like Haier, is part of the ongoing transformation efforts at companies like IBM and Procter & Gamble, and is a top agenda topic in executive forums on strategy and organization design.
In all these areas, the commons is still emerging: its seeds are scattered, progress is fitful, and open practices are often seen as fringe challengers to the mainstream. As always occurs at the early stages of grand transformations, it’s easy to point to a long list of roadblocks, but a bit of imagination and reflection about the recent past make an “open everything” future seem not only possible, but economically inevitable.
Indeed, in addition to the positive feedback inherent in each of the four domains above (software, research, designs, and collaboration), they reinforce each other as well: just as software has unlocked massive transformations in science, technological development, and teamwork, we’ll see those increasingly feeding back into each other in myriad ways, creating what Peter Diamandis calls “convergence”, a combinatorial explosion of exponential feedback loops, generating hyperexponential growth.
For instance, consider the field of integrated circuit fabrication, dominated for decades by specialized giants like Intel and AMD. Can open research into materials and nanotech, in combination with ML-driven chemical modeling, bring this field back into the realm of open development, where it began? Might the next revolution in computer hardware come about as a product of dedicated engineers working across multiple companies, rapidly prototyping and experimenting together on — and feeding back into — an open platform?
The 2020s: A typical “buildware” development journey
If the airplane scenario sounded a bit crazy, we’re about to crank it up a notch. Say a few years from now you move into a new building, and are dismayed to find it doesn’t have a green roof. You log into your building management app, click the “Advanced” tab, and find a full-featured “buildware development environment”, where all tenants have complete access to “look under the hood” and propose changes. After checking out the building’s specs, you do a quick search and find a few turn-key green roof components, complete with bills of materials, deployment instructions, and embedded hardware to take care of maintenance. You also find a tool that looks up your building by address on an open repository of specs and operational data for all the world’s buildings that have opted in, and optimizes parameters such as soil composition and plant mix depending on your cost target, weather stats for your geographical location, and the available area to build.
Being a bit of a buildware geek, you look up a couple of examples online — from buildings with live green roofs of different types, as well as examples that went wrong. In a couple of hours, you’ve figured out how to wire up the new component to the building specs. The platform asks you to run automated tests that ensure the modified design will work, then you publish it for the other tenants and the management company to review. After a few modifications and final approval, you hit the “Deploy” button and are offered an automatically generated list of contractor quotes that can execute the design. They’re all quite reputable and many of them have worked together, as you can easily see from an automatically generated trust network diagram. You and the other tenants choose one that’s a bit more expensive and new in the business, but is using latest-generation drones that automate 90% of the construction; you know she’ll be publishing all her test findings back into the buildware commons, which gives her access to extra government greening subsidies.
Another quick round of project reviews with the contractor and the requisite automated regulatory approvals, and you send over the contract, which gives her read/write access to the building areas and systems. The next morning there she is, drone army in hand; by the end of day the roof is fully deployed, tested and certified. You go back to the app and it runs a quick project retrospective:
- Total cost: less than $10,000, and mostly subsidized by your city’s greening program, the money paid directly to the contractor as soon as the certificate is generated.
- Total time spent: 2 days from you as “project leader”, plus a few minutes or hours from the rest of the building stakeholders.
- Total value generated: one more green roof to help reduce your city’s environmental footprint; one more example in the buildware commons, accompanied by live data to help others benchmark performance; and one more validation of the new contractor and her new methods, helping her build up credibility and improving the skill commons. Oh, and of course, the private benefits for the building tenants — reduced HVAC costs and lovely native vegetation on your rooftop terrace.
After this rewarding experience, you read more about open buildware and learn that it’s not only for green roofs, or even just for individual buildings: whole cities and nations are leveraging this ecosystem of data, scientific models, component designs, and collaborating companies to tackle reforestation, ocean cleanup, grid decentralization, catastrophe response and many other challenges, much faster, cheaper and more effectively than they could ever do before.
You start liking this space so much that you decide to expand your new buildware skills. Suddenly, you have an idea for a new startup…
The future is open and positive-sum by default
The previous example was a bit long-winded, but I feel it’s worth going into the details to get a sense for how this whole hyperexponential will play out in practice, not just in the abstract. Indeed, each “magic moment” in that story is technically and economically feasible — many are already possible today. Even the idea of an automated regulatory system based on open standards, which reads like a pipe dream to the majority of us who live in red tape-obsessed jurisdictions, is not so far-fetched in the context of “smart cities” and e-gov projects.
The grander challenge is assembling the whole ecosystem. Today, VCs ask startups to prove “one miracle at a time”; this is fine for localized, incremental validation, but to build the kinds of wide-spanning systems we’re talking about, this philosophy needs to be coupled with long-term vision and sustained purposive effort.
Which leads back to the points made in my previous article. As parts of the great supermind called humanity, we are all incessantly finding ways to improve the system, i.e., to make the supermind smarter. Most of those improvement opportunities, however powerful, are localized to specific “applications” (regions, industries or domains); in contrast, the shift to the “principle of openness” is a operating system upgrade, vastly expanding the bandwidth and speed of our collective learning and development processes, allowing us not only to solve today’s grand systemic challenges, but to implement radically new capabilities that would have been unthinkable before. Socioeconomic and technical advances will continue to feed into each other, giving rise to an explosive and unpredictable flowering of sudden-seeming advances in both realms. Indeed, in the near future, not only our gadgets but also our “social technology” will look like magic to those of us who are around to witness it.
Humanity has done impressively well on many fronts, but our social, economic and organizational infrastructure is still the equivalent of monolithic, disconnected 1960s mainframes; no wonder we have a hard time solving global problems. What will the human equivalent of the Internet of Things look like? Who will lead the way? How soon until we get there — and then where do we go next?
