The (broken) economics of OSS

Last week, Redis Labs announced a change in how they will license some extensions to the core Redis project (see here and here for more information on the change). This prompted substantial dialog about the efficacy of the new license as it relates to Redis the OSS project as well as Redis Labs (the business trying to generate revenue on top of the OSS).

In response I wrote this Twitter thread:

Paul Dix, CTO of InfluxData, then followed up with an excellent blog post which I highly recommend reading. I responded on Twitter with another thread:

Given the above background, I want to provide a long form discussion of my two Twitter threads as this topic is nuanced and quite interesting.

Note: this post is heavy on opinion and light on facts/references backing up those opinions. Thus, preface everything that follows with “IMO.”

The history of OSS

In the early days of computing, primarily driven by academics, much of the intellectual property created was born out of the belief that all software should be free and open. During the late 70s and early 80s, proprietary software licensing took hold (driven by IBM, Microsoft, Apple, and others), and with it, the parallel free software movement that brought GNU, Linux, and eventually most of the software we use today in modern infrastructure computing (see Wikipedia for a more detailed history).

It’s important to point out that the early days of the free software movement were driven primarily by hobbyists and “true believers.” These developers donated their often considerable time to openly collaborate on various pieces of core functionality (compilers, linkers, operating systems, etc.) that would later enable an entire ecosystem of free software to be built on top.

Modern expectations: software is free and open

Since the early 90s, and primarily driven by Linux, there has gradually been a mental shift among those in the high-scale computing industry: proprietary software should be avoided at all costs. Why?

1. Open software quality has vastly increased over time. This is a key and subtle point. As mentioned briefly above, the origins of the modern free software movement were driven primarily by hobbyists and professional developers donating their time during nights and weekends. From a quality and functionality perspective, how could this group compete with Microsoft, Apple, Oracle, IBM, and others during the 80s and early 90s? They could not. However, the late 90s brought with it Yahoo, Amazon, and Google. The 00s brought Facebook, Twitter, and others. In short, starting in the late 90s, massively successful companies that were not directly generating revenue from software or hardware started driving forward many of the most popular free software projects by hiring professional developers (often from Microsoft, Apple, and other proprietary software companies) and putting them to work full-time on OSS projects. Thus, over time, the workforce contributing to the most popular free software has gradually shifted from hobbyists to well paid and highly skilled professional programmers subsidized by the profits of Internet companies.
2. Open software allows modification. What if a proprietary software company goes out of business? How will users continue to make changes to meet their business needs? What if users need to make a change that the company does not feel is valuable and refuses to prioritize?
3. Open software avoids cost lock-in. As computing has shifted progressively away from “shrink-wrapped” software to services, often run across thousands or even millions of computers, the largest Internet companies do not want to be locked-in to per-node, per-CPU, etc. pricing for critical systems that they depend upon such as hypervisors, operating systems, databases, load balancers, etc.

The previous points have led to a place in which software that is not free and open is treated with skepticism.

Why do developers create OSS vs. proprietary software in the modern era?

The computing industry has gradually shifted from generating revenue directly from software to generating revenue from value added services built on top (often supported by advertising). This is not to say that propriety software no longer exists. Clearly it does in the form of consumer applications (games, Windows, OSX, Android, apps, etc.). In the enterprise space it also still exists in the form of line of business applications (although I would argue that most large LOB applications are now custom developed in-house by the company that needs them). However, when focusing on the infrastructure (cloud) computing space, there is a dogmatic feeling throughout the industry that core tooling such as compilers, operating systems, databases, load balancers, etc. should be free and open.

Within the infrastructure space, given that the bias is now towards software being free and open versus proprietary, what are the reasons for creating new OSS in the modern era? There are a few:

1. Hobbyists. The simplest (and most noble) explanation for creating OSS remains that a developer (or group of developers) sees a problem domain they feel is undeserved by software (or served by a proprietary offering they don’t want to pay for) and decides to start a collaborative process to implement a solution. This is no different from the early days of the free software movement.
2. Internet mega-company byproduct. As described above, companies like Amazon, Google, and Facebook do not believe in using proprietary software for critical infrastructure computing functions. In addition to bolstering previously existing OSS, they have also started many critical projects and released them as OSS. Why? Isn’t the software they develop a competitive advantage even if not domain specific? Let’s be clear that Internet mega-companies do not do anything out of goodwill (Google for example is notorious for releasing whitepapers about internal infrastructure technologies but not the software itself). Releasing OSS is a calculation that incorporates recruiting (developers want to work on OSS as it gives them industry visibility), technical credibility (helps with recruiting and the perception of the company as a technical powerhouse), and platform building (the industry orienting around a set of OSS technologies driven by an Internet giant provides a substantial advantage when offering related cloud computing products).
3. Mind-share building for an infrastructure computing startup. There are still quite a few VC funded infrastructure startups forming every year. In the modern era, it’s essentially inconceivable that such a company can find success with a proprietary software offering. Instead, companies start with OSS, hope to gain mind and market-share, and then pivot to figuring out how to generate revenue on top (more on this below).

It’s important to point out that the above OSS creation options are not mutually exclusive. For example, software can be started by a hobbyist and then move under the umbrella of an Internet giant or a VC funded startup. Or software can be created by an Internet giant and then key maintainers leave and start a VC funded company.

Who pays for OSS?

Nothing is free. All created OSS is paid for somehow. For each of the OSS creation reasons described in the previous section, let’s look at how the software is paid for.

1. Hobbyists are paying with their time. This is time that they could be spending earning a living building proprietary software, spending with their family, engaging in recreation, etc.
2. Internet mega-companies are subsidizing OSS creation with their primary revenue generation stream. For Google, search and advertising. For Facebook, advertising. For Apple, iPhone. For Lyft, ride sharing. The list goes on.
3. Infrastructure computing startups are subsidizing OSS creation with VC dollars. VCs invest under the assumption that once an OSS project attains popularity, a revenue generating business model can be built on top.

It’s important to note that other than hobbyists and true altruists, the Internet companies and VC-backed startups all have ulterior motives for releasing OSS as described in the previous section. To believe otherwise is foolish.

OSS business models for startups

I’m not going to describe in depth the various ways in which startups are attempting to make money on OSS in the modern era. For more information see this blog post by Paul Dix as well as my own post on why I decided to not start a company around Envoy. However, I will briefly describe them here in enough detail that the rest of this post will make sense in isolation.

1. Open core. The definition of open core is not well established, which creates substantial miscommunication when discussing this business model. However, the general idea is that some subset of features in the OSS are placed behind a paywall. If users want to consume those features they need to pay.
2. SaaS. In this model, the OSS is free, but a company will run it for the customer (either in the cloud or on-prem) for a fee. The idea is that operating a sophisticated piece of software 24/7 is non-trivial and worth paying for. SaaS can also mean the layering of additional functionality on top of the core OSS product (e.g., UIs, management systems, analytics engines, policy tools, cost analysis, etc.). Thus, the line between open core and SaaS is blurry, a point which I will discuss in depth in the next section.
3. Consulting, services, and support. The OSS is free, but one or more companies are formed to provide consulting, feature creation for hire, and general support, all for a fee. This business model is what I would call a “lifestyle business.” It may generate enough revenue for a comfortable lifestyle, but it is extremely unlikely to generate enough revenue to excite the VCs.

If we take consulting, services, and support off the table as an option for high-growth revenue generation (the only thing VCs care about), we are left with open core, SaaS, or some blurring of the two.

What is open core really?

This is where things start to get interesting. There is no single definition of open core. After the initial Redis story broke, Salvatore Sanfilippo (creator of Redis) wrote a blog post laying out his case that Redis had not switched to an open core model. His argument can be reduced to the fact that Redis is a modular system, and that only modules developed by Redis Labs would be under a commercial license. The Redis core will remain fully open. He goes on to write:

In an open core business model around an open source system it is *fundamental* that you take something useful out of the free software part.

At a high level, I agree with Salvatore that this is the most basic definition of open core, however, I do believe that Redis Labs has fundamentally switched to an open core business model, and the implications of this switch are subtle and gradual.

The problem with “pure” open core

Whether an OSS project is modular or chooses to release commercial/enterprise closed source builds with extra features, there are some fundamental problems with the “pure” open core model:

1. Open core is at odds with the OSS community. What happens when a community member wants to contribute a feature/module that competes with the enterprise offering? How should the company handle that situation? They are in no-win scenario. If they refuse to take the contribution they alienate the community. If they take the contribution they cannibalize their own business model.
2. Cloud vendors and large Internet companies have the resources to take the core and build the rest. Large cloud vendors are well equipped to build managed SaaS offerings on top of popular infrastructure OSS. Even if certain pieces of critical functionality are missing from the OSS offering, if important enough, the features can be rebuilt with enough resources. Whether this rebuilding process is actually worth it or not is a complex equation and the subject of the next section, however at the scale that the “big three” clouds operate, it probably is worth it. As the inevitable march to the public cloud continues over the next 10–20 years, selling open core OSS independently to enterprises will become an increasingly tall order.

Our industry’s inability to accurately evaluate Total Cost of Ownership (TCO)

Although it’s been implied throughout this post, it should be explicitly stated that many (most?) software build versus buy decisions are subjective and made without any real data. All too often, potential customers feel that infrastructure software should be free, and grossly underestimate the true cost of running a piece of software over time, when taking into account both CAPEX and OPEX. Theoretically, the CAPEX for OSS is $0. But what is the OPEX? How many engineers will have to be employed to run the software and debug it? What features will have to be added to an unfamiliar code base? What additional ancillary management systems, UIs, etc. will need to built such that the software can be consumed by the rest of the organization?

If our industry was better at analyzing true TCO, I fully believe that making money off of OSS would be substantially easier. The experts in the software would be able to easily charge substantial per-node licensing fees in exchange for continued feature development and support to all but the largest enterprises and cloud providers. The fact that the software is free would really just be an insurance policy in case the company goes out of business.

Unfortunately, software buying decisions are made by humans, not machines, operating with imperfect information and biases on evaluating true cost. Thus, we have a logjam:

Everyone wants infrastructure software to be free and continuously developed by highly skilled professional developers (who in turn expect to make substantial salaries), but no one wants to pay for it. The economics of this situation are unsustainable and broken.

The advantages of “loose” open core and SaaS

Having laid out the problems with “pure” open core and the imperfect decision making process that our industry uses when deciding whether to buy or build software, we now come to what I have recently called “loose” open core and SaaS.

In the future, I believe the most successful OSS projects will be primarily monetized via this method. What is it?

The idea behind “loose” open core and SaaS is that a popular OSS project can be developed as a completely community driven project (this avoids the conflicts of interest inherent in “pure” open core), while value added proprietary services and software can be sold in an ecosystem that forms around the OSS. What are some examples of such services and software?

• Modern “cloud native” infrastructure software is robust, but typically configured via a giant pile of YAML. Custom UIs that allow organizations to manage configurations are extremely powerful. When coupled with RBAC, version control, etc. they become even more so.
• Analytics tooling.
• Security analysis and policy enforcement tooling.
• Observability and monitoring tooling.
• Hosted and on-prem managed SaaS offerings. Operating software at scale is extremely non-trivial.
• Domain specific features that can be implemented as add-ons without requiring any core modifications. For example, DDoS detection and mitigation tooling using Envoy’s OSS APIs and filters.

The other benefit of “loose” open core technologies are that they are more difficult for the large cloud vendors to subsume. Historically, the clouds are very good at creating basic managed SaaS offerings of popular OSS. However, they have been much less effective at building the “special sauce” on top. If executed well, a robust “loose” open core business can be built that is compatible, not competitive, with the cloud offerings.

Project popularity and business models

I had an interesting insight recently: I don’t think there is a one-size-fits-all business model for every OSS project. In fact, I think the best business model is dependent on the project’s popularity and might change over time. Why would that be?

New projects

For new projects being built with VC funding or by hobbyists, and without the ability to battle-test the software in the production environment of a large Internet company, the larger software community will remain skeptical of project quality for quite some time. These projects need to get a toehold in production wherever they can, and initially will rely on word of mouth, conference talks, Twitter, Hacker News, etc. As a project gains some traction, I think it’s perfectly reasonable to target a “pure” open core business model coupled with services and support in order to fund continued development (of course “loose” open core and SaaS are fine too but might require more development resources than the startup has). In fact, at this stage, I would argue it’s more likely that some small number of early adopters will pay for a “pure” open core product and/or a services and support contract, satisfying the relatively small amount of growth that VCs want to see during early funding rounds.

Popular projects

Unfortunately, there is an inflection point at which in some sense an OSS project becomes too popular for its own good, and outgrows its ability to generate enough revenue via either “pure” open core or services and support. How?

As a project reaches the upper echelon of popularity and number of production deployments, the community surrounding the project by definition will have commensurately increased in size (including most likely the very largest technology companies). Thus, the likelihood someone in the community is going to want to add features that directly compete with a “pure” open core business model, and refuse to purchase the enterprise version of the software that contains the feature, becomes much more likely. At this point, the project and surrounding community starts to become strained as a conflict of interest kicks in between project governance and business interests.

At this point, the community alienation makes it much more likely that an upstart OSS competitor will form (built by someone else who either doesn’t want to pay for the enterprise edition of a different project, or built for a different but related reason and happens to subsume the role of the competing enterprise product). This point is far from academic. In fact, it is the driving force behind Envoy’s rapid adoption in the industry.

A summary of the problems

As was discussed previously, as much as everyone would like to break the laws of economics, OSS is not free. Furthermore, modern OSS is big business, whether produced by the large Internet companies or VC-backed startups. The days of basement hacking producing the most popular projects are largely gone. What we are left with is a set of substantial problems that do not have a good answer:

• Everyone wants well-designed, bug-free, and feature-rich free software, but no one wants to pay for it.
• Very popular OSS projects started by the large Internet companies may appear to be community driven, but there are significant ulterior motives at play.
• OSS projects started by VC-backed companies are struggling to figure out how to monetize their creations. “Pure” open core may work initially, but if the project becomes popular enough, community alienation will become a substantial problem when deciding which features to accept in OSS, and conflict with the ability to reach the revenue targets that VCs are looking for in later funding rounds.
• Basic SaaS offerings by VC-backed companies are easily subsumed by the major cloud vendors.

From my perspective, since it seems unlikely that we are going to change the psychology behind how our industry evaluates TCO, and if we want to stay optimistic that the cloud vendors will not end up eating everyone’s lunch, building a vibrant community and then enabling an ecosystem of “loose” open core and SaaS businesses on top appears to me to be the only viable path forward for modern VC-backed OSS startups.

Maintainership, software foundations, and fellowships

If we assume that the path forward for revenue generating OSS is a vibrant community and an ecosystem of companies built on top, it becomes extremely important to talk about the maintainers. Who pays the key maintainers? How do they stay impartial such that they allow vibrant companies to form around the software in a way that does not favor any one company? How are they compensated such that they can earn a salary roughly commensurate with what they could earn if they worked for a major tech company either on OSS or on proprietary software and systems?

Today, we unfortunately do not have consistent answers to the above questions. Key maintainers currently operate under an almost feudal system of patronage, hopping from company to company, trying to earn a living, keep the community vibrant, and all the while stay impartial (which I would argue is almost impossible to do if earning a salary from a company that has a commercial interest in the OSS project).

One possible solution to this problem is for the various OSS foundations to step up and start providing fellowships to key maintainers. The foundations already raise substantial amounts of money from members to support their activities. It’s a natural progression for the foundation to raise money directly for fellowships to support the work of key maintainers on key projects. And I think it is a win-win for everyone involved. Here is why:

1. By allowing key maintainers to stay independent, it becomes much more likely that a project’s community remains robust and does not favor any individual corporate entity. This creates a larger community and a more vibrant ecosystem for “loose” open core and SaaS companies to be built on top.
2. By funding fellowships, companies that utilize the OSS for revenue generation ensure that implicit project governance does not inadvertently fall into the hands of a competitor.
3. Finally, because the fellowships are likely to be funded primarily by large Internet companies, they can pay well, with salaries commensurate with what the maintainers would earn elsewhere.

In short, everyone wins.

The future of OSS

It’s very difficult to foresee how the future of revenue generating OSS is going to play out over the next 10–20 years. Will most core infrastructure software be subsumed by the large clouds making all of this a moot point? Will VC-backed OSS startups continue to find success operating independently?

In any case, the march of OSS across the industry will not stop, that is a given. Because of this, as an industry, we are going to have to come to terms with the economic reality: nothing is free, including OSS. If we want vibrant OSS projects maintained by engineers that are well compensated and not conflicted, we are going to have to decide that this is something worth paying for. In my opinion, fellowships provided by OSS foundations and funded by companies generating revenue off of the OSS is a great way to start down this path.