Theodore Ts'o
8 min readNov 2, 2023

--

Open Source in Shambles? Not so fast.

As an open-source contributor --- I was the first North American Linux Kernel hacker, who started contributing to Linux in 1991, and for whom the majority of my career, my salary has come from my open source work, I have a bone to pick with those who claim that "Open Source is dead," or "Open Source is in shambles", or "Open Source is Struggling".

My open source career has spawned the four employers. The first, at MIT, was when I was contributing to the Linux kernel as a hobbyist, on my own time, although I did get support from MIT when they allowed me to run the first North American software distribution point using spare CPU cycles on my desktop computer at MIT (and later, tsx-11.mit.edu was allowed to use space in an MIT machine room and of course, MIT donated the networking bandwidth ---- just as MIT donated networking bandwidth for the Free Software Foundation's distribution of GCC, GNU Emacs, and a large number of Open Source packages.)

My second open source employer was VA Linux Systems, which was a startup launched during the dot-COM boom. There, I learned many things about how not to run a business. For example, even though Venture Capitalists only care about revenues numbers growing exponentially --- "up and to the right!" --- if you are selling hardware at $50 dollars less than the Bill of Material costs, not including the money needed for R&D and assembly costs, selling huge amounts of hardware to customers like Akamai at a loss, and making it up in volume, this isn't a formula for long-term success. This is true even if your comapny achieves IPO and your VC's were able to claim a successful exit with a record setting one-day post-IPO stock price pop.

My next open source employers were at IBM and Google, and at these companies I've learned a lot more about open source strategy. I can say with some authority that the claims made by Jan Kammerath are vastly over-simplified. The Linux kernel is not a solo project; nor is it an underfunded survivor; nor is it "actually commercial software"; nor is it a FAANG project, started by a FAANG organization, entirely funded by FAANG companies, and controlled by FAANG.

And you can see that by looking at the Linux Kernel regular development statistics. We track the number of code contributors, the number of new contributors in the past 3 month development cycle; the number of companies whose employees have contributed to the Linux kernel, and who the top contributors to the Linux kernel are, by individual, and by company. For example, the most recent set of development statistics can be found here[1]. For the previous Linux kernel release, 6.5, see [2].

[1] https://lwn.net/SubscriberLink/948970/1632e7b63a906ab0/

[2] https://lwn.net/Articles/941675/

The Linux 6.6 kernel is the last kernel release for calendar year 2023, so the statistics are cover the range of the Linux 6.2 through 6.6 kernels. During that period the top 10 contributors by company were: Linaro (11.7%), Red Hat (11.2%), Google (11.1%), Intel (10.8%), Meta (8.3%), Qualcomm (6.1%), AMD (5.5%), Linux Foundation (5.1%), SuSE (3.9%), NVIDIA (2.7%), and Oracle (1.9%). This is followed by "Unknown" (1.8%), where we don't know the corporate affiliation, if any, and "None" (1.8%) which are people who are hobbyists or college students. "Consultants" (1.4%) may be paid by companies such as Google, Meta, or other large companies, but that's a pretty tiny number.

This is what a healthy open source project looks like, and so this is an existence proof that they can very much exist, and they don't have to be a solo effort, or one where a single company completely controls the effort.

Now, every single company has to have a valid business reason for contributing to an open source project, and not all open source strategies are equal in terms of business validity. If a company doesn't have a valid business model (for example, such as one of my previous employers, VA Linux Systems), those contributions, or perhaps the company's existence, will not be long for this world --- even if it initially was planned to IPO at $12/share, actually IPO'ed at $30/share, and closed the first day at $239.25/share, with a market value over half that of Apple Computer. Reality has a way of catching up with companies, even if they have really good Investment PR teams.....

One such Open Source business model is to sell service and support. This model certainly works, so long as you are careful not to over-extend yourself. An example of this would be Cygnus Support, which sold support for the GNU C Compiler and related tools. The problem is that the growth of such a company will cause Venture Capitalists derogatorily refer to it as a "Lifestyle Startup" --- one that makes a good product, keeps its customers and employees happy, but which isn't growing at an exponential rate in order allow the VC's to make Bank and become wealthy beyond our (although perhaps not their) wildest dreams. So this model is great for the open source ecosystem, but it's not so great for those who are interested in making our income inequality in the world even worse than it already is.

Open source business models which allow companies (and their VC investors) to make serious bank require there to be some kind of unique value proposition that can't be cloned by a competitor, allowing that company to extract significant amounts of money.

Sometimes this can be a software patent (although this model is one that is a bit... controversial... in some quarters). Others might involve hardware, which can't be easily cloned by a competitor (perhaps because that hardware has hardware patents, or just because developing a hardware product requires a significant investment and lead times of 18 to 24 months in some cases). Or it might be that a company might contribute a relatively small amount of code to an existing open source project (say, for a plugin, or a device driver, or a new file system feature), which enables a much larger investment, which might be a Cloud Infrastructure (as sold by Amazon, Google, or Microsoft), or might enable a piece of hardware or CPU, or which enables a much larger set of code that might be proprietary in nature.

For example, consider Kubernetes or other projects which are part of the Cloud Native Computing Foundation. Yes, Kubernetes was originally developed by Google. But if you look at who actually contributors to Kubernetes in the last quarter[3], Google's contributions is less than 18%. Other contributors include VM Ware, Red Hat, Microsoft, Amazon, DaoCloud, The Scale Factory, and Intel. The steering committee has representatives from Red Hat, DaoCloud, Intel, Cisco, Google, and VMWare, with Google holding only 2 out of the 7 steering committee seats. This is hardly an open source project controlled and contributed solely by FAANG companies.

[3] https://k8s.devstats.cncf.io/d/9/companies-table?orgId=1&var-period_name=Last%20quarter&var-metric=contributions

Google certainly derives business value from Kubernetes; it has a cloud product, Google Kubernetes Engine, which has a large number of customers. Obviously, all of that cloud infrastructure control plane is proprietary code, and it would be non-trivial for a competitor to duplicate it. But both Amazon and Microsoft have, with Amazon EKS, and Microsoft AKS, respectively. IBM also has a Kubernetes service offering.

Crucially, it was important that Kubernetes be open sourced, because it meant that customers could adopt Kubernetes without fearing that they would get locked into a single vendor's cloud infrastructure. Google had a first-mover advantage when they open sourced Kubernetes, and that had enough business value to justify the open sourcing of that whole project.

I could go on and on with other models, such as that of Android, the Chromium web browser, ChromeOS, and so on. All of these are full open source projects, but they also have some proprietary add-ons that are used in the commercial version of these code bases. And that's because it's fair to assume that people won't donate money just out of their goodness of their own hearts. If that was your business model, it's about as flawed as VA Linux's model of selling servers for $50 under the cost of their components. But just because some business models associated with Open Source are flawed, it doesn't follow that all Open Source business models are flawed.

I leave you with this. As an Open Source developer, I really don't care how many users use code that I put out in the world. It does have some benefit, in that it might enable other companies to consider it valuable to contribute to my open source project, and it might make it more likely that a company is willing to pay me as an Open Source maintainer. However, ultimately, the value of having large numbers of users, from an Open Source perspective, is only a 2nd- or 3rd- order effect.

No, what's really important is whether other people are willing to contribute code (not money!) to my subsystem or my open source project. That's what takes the project forward, and if those other people happen to work for other companies that implies that those companies have figured out a way of extracting business value from my open source project, and in order to extract more value, they are willing to contribute new features or capabilities or improve the performance of my subsystem or project. When they do that, it leverages the value that my subsystem or project can deliver to my employer, and makes it more likely that my employer will be happy with my work. So as the ext4 file system maintainer, with my open source maintainer hat on, one of my important jobs is to make it easier for other company's developers to contribute code that enables their company's business goals, because in the long term, that will help enable my company's business goals as well.

This is the moral of the folk story of "Stone Soup"[4]. However, unlike in this fable where the travelers "tricked" the villagers into contributing all of the food to make a tasty soup that all can share, the difference is that with Open Source Software, we can let other people share in the fare without diminishing the value to the people who have contributed to making the Stone Soup. But that's also why the freeloading "problem" doesn't bother me as an Open Source contributor. If you use my software, but don't pay me, that's fine. But you're also largely irrelevant to me. If you want someone to develop a feature for free; then yes, you don't understand how things works in the Open Source world.

[4] https://en.wikipedia.org/wiki/Stone_Soup

What you can do, however, is to implement that feature yourself, or pay someone to implement that feature for you ---- and unlike with proprietary software, you know that you can make those changes (good luck trying to add some new feature to Microsoft Word or Oracle Enterprise Database!), and if that work is done well, if you choose to contribute it, it will be accepted as a contribution for all others to share, and when new versions of the open source project get released, you won’t have to constantly reintegrate those changes into the open source code.

--

--