FOSSY panelists talk rights; what about responsibilities?
On July 14, the FOSSY conference hosted a RHEL Panel Discussion, with Bradley M. Kuhn of the Software Freedom Conservancy, benny Vasquez of the AlmaLinux OS Foundation, James Wright of Oracle, and Jeremy Allison of CIQ participating. I was interested when I learned about it, but disappointed when I wasn’t able to find a recording. As it turns out, the panel had been recorded and I simply hadn’t found it until last week.
There were a handful of statements made during the panel that I think deserve fact-checking, but that might not be the most interesting bit, so I’ll lead with those and try to separate that from the commentary. Skip a bit if you tend to think fact checking is pedantic.
Fact checking:
At 1:00 in the recording, Bradley said: “For the last 20 years, a company named Red Hat owned by IBM has had a business model that has involved subscription services to a Linux based operating system. Historically the source code for this was fully made public.”
Fact: RHEL source code has not been fully public for a very long time, if ever. It’s difficult, today, to find detailed information about the minor release life cycle of early RHEL releases, but Wikipedia records “Extended Lifecycle Support” dates for every RHEL release back to version 3, which is the first release that was rebuilt by CentOS. Red Hat has never published to the public the source RPMs for updates that they ship to customers in the extended lifecycle support phases.
This is easier to visualize for current releases, for which Red Hat provides diagrams of the life cycle as planning guides. There, you can see that RHEL minor releases overlap in order to provide users with continued support as they update from one minor release to the next on their own schedule, and to support customers who need a feature-stable release for more than 6 months. The maintenance phases that provide that support are not published to the public, and they have never been included in CentOS or any other rebuild.
That is to say that Red Hat has always published source packages only for the latest release in a major series, and not for the other minor releases that they continue to maintain. Today, the latest release branch is CentOS Stream.
Around 10:30, Jim says: “The Free Software Foundation’s Free Software definition requires that the software be freely redistributable. The OSI’s Open Source definition requires that the software be freely redistributable, and that the source be published.”
Fact: Both definitions require redistribution to be allowed, but neither the Free Software Definition, nor the Open Source Definition requires a vendor to publish source code to the public.
At 14:20, Jim says: “They say people who are rebuilding don’t add value.”
Fact: Red Hat does not claim that the people rebuilding RHEL don’t add value. They’ve said that rebuilding RHEL isn’t the thing that adds value.
Red Hat’s statement about the value of a rebuild is not an attack on the people who rebuild, it is a statement about the act of rebuilding, alone.
This is definitely commentary, but as a labor advocate, I will always stress that labor creates value. Developing software creates value. Fixing bugs that affect your customers creates value. Merely rebuilding a source code archive is not a creative act; it is a purely mechanical one. A machine takes an archive as input and transforms it into an executable form. That process doesn’t create value that did not already exist.
Presenting Red Hat’s statement as an attack on the people may energize and unite people around your cause, so I can see why rebuilders present it this way, but it is manipulative and untrue. (Much like the false claim that Red Hat has accused rebuilds or their users of being “freeloaders.”)
Commentary:
Selling an “enterprise support” contract is fundamentally incompatible with the goal of strictly rebuilding source without modification.
Anyone who has been either a customer or a vendor in an enterprise support agreement knows that one component of the agreement is technical support for the product. Customers who encounter an issue will typically open a ticket first with tier 1 support, who can answer basic or common questions, and triage questions which require the expertise of more experienced engineers. Those questions will be escalated, and eventually they may reach engineers who actually develop the product, who can make determinations about whether product behavior is a bug, and who can modify the product in order to provide a fix to the customer.
That tier does not and can not exist if the explicit aim of the vendor is to merely rebuild software and not to make any changes to it. The claim of “bug for bug” compatibility is implicitly a statement that no bug will be fixed in a derived product unless Red Hat fixes the bug. For any bug that exists in a rebuild, either the customer or the rebuild vendor will have to escalate that class of issues to the upstream vendor, offloading responsibility for providing support to the affected customer.
That arrangement is exploitative, and should be recognized as such. It is dishonest to offer “enterprise support” for a product that you don’t intend to develop.
This contradiction can be resolved by either dropping the claim of “bug for bug” compatibility and committing to actually developing a derived product, or by dropping claims of “enterprise support,” which fundamentally requires that a vendor be able and willing to fix bugs when they are found.
It happens to all of us: you’ve confused “Free” (as in speech) with “free” (as in beer)
Several times in the panel, Jeremy makes the point that “Customers want Freedom,” and because he is using the word “freedom,” I think the only rational inference is that he means liberty, and his implication is that rebuilding Red Hat’s source code creates liberty that does not exist otherwise.
That is backward reasoning. Derived distributions are possible because the software is Free. They’re proof that it was Free to begin with. If the software had not been Free, it would not be possible to build and redistribute it.
Rebuilding Red Hat’s source doesn’t create something more Free than the original, just something that’s free (as in beer).
That’s an important distinction, because in our passion for Freedom, we may seek to make a product Free and instead merely make it free, which is not actually a moral good.
Make-work
Jeremy also repeats several times that Red Hat’s discontinuing their explicit support of rebuilds by discontinuing the process of de-branding and publishing source for branched releases doesn’t prevent them from building derived distributions, calling it “make-work.”
The problem with that argument is that the entire process of rebuilding RHEL is make-work.
Customers who want an OS that is ABI compatible with RHEL, that’s free-as-in-speech and free-as-in-beer can get that, without limitation, in CentOS Stream.
Developers who want to build and test their products on RHEL may be able to get that as Red Hat Developer Subscription for Teams.
If you believe that there is something unsuitable or risky about Stream, attempting to rebuild RHEL is the most laborious solution to the problem imaginable. Are you concerned that a bad update will slip through Stream and RHEL’s QE? Providing your customers with a managed update repository that provides a canary environment and a feedback process to detect and defer regressions would be vastly more valuable and less laborious than rebuilding RHEL. Do you want a 10 year life cycle? Supporting Stream for its 5 year cycle and then providing your own update repository for security and critical bug fixes after its 5 year cycle would be less laborious than attempting to rebuild RHEL for its full 10 year cycle. It is disingenuous to complain about “make-work” when you have selected the most laborious solution available for your problem.
Even if we accept for the sake of argument that a rebuild of RHEL was needed, it’s not at all clear that there’s a need or a benefit from having multiple strict rebuilds without modification, which appears to be the intention of the Open Enterprise Linux Association which was recently announced. All of you would have less work to do if you cooperatively built one binary distribution and individually sold support for it. Isn’t rebuilding RHEL, repeatedly, “make-work”?
If we’re honest and consistent, “make-work” goes all the way up to developing a product that’s available as part of a subscription agreement and not given away for free in executable form. The only way that Red Hat could not be accused of make-work is by simply giving the software to everyone who wants it, and frankly that isn’t required by the license. They’re allowed to make and sell a product. Nothing in the license requires them to give it away for free.
And honesty further requires us to confront two fundamental truths that are being ignored by this panel.
The overstatement
The panelists consistently discuss the matter of RHEL’s licensing and subscription agreement in terms of their compliance with the GPL, but the GPL doesn’t cover the majority of RHEL. Something like 1/3 of the packages in RHEL are covered by a GPL version, with most of the remaining 2/3 of the components being covered by an MIT, BSD, or Apache license. Those who have discussed the advantages of the GPL vs those more permissive licenses with groups of developers know that developers who chose permissive licenses do so not only knowing that their code can be included in products with more restrictive terms, but explicitly to allow software vendors to sell software that isn’t given away for free.
The majority of software in RHEL was developed by people who explicitly chose to support models like RHEL’s.
And while I believe that Red Hat’s subscription agreement is entirely compliant with the GPL, I think the fact that Red Hat can place redistribution limitations on the majority of the distribution, coupled with the fact that unlike models like “open core” the code for all of the features of RHEL are freely available) makes the question of whether their subscription agreement is GPL compliant to be un-interesting.
Unless derived distributions are interested in publishing an OS that provides the GPLed 1/3 or less of RHEL (I don’t know how many GPL packages have non-GPL dependencies), they are significantly overstating the extent to which the GPL protects their access to RHEL’s source code. Similarly, because Stream is the full and complete source code for RHEL’s major-release branch, they are dramatically overstating accusations that access to the code has been withdrawn to begin with.
The omission
The panelists spend a lot of time talking about their rights, but we need to also acknowledge that Free Software development comes with responsibilities. One of those responsibilities is using trademarks in a manner consistent with guidelines published by the trademark owner.
This should not be a controversial point of view. While the Free Software Foundation is concerned with the right of users to use, modify, and distribute software, they are also proponents of the rights of trademark holders to decide how their trademarks are used.
Red Hat publishes a guide for trademark use which says, among other terms, “You may not name or brand your product “Red Hat Linux,” “Red Hat Enterprise Linux,” or use the Red Hat trademarks in any way, either on your product or in advertising. You must use a different trademark for your product that will not cause confusion with the trademarks of Red Hat or another party, will not indicate or imply that your product originates from or is sponsored or approved by Red Hat.” (emphasis added)
A great deal of time has been spent talking about the spirit of open source and the spirit of the GPL. While it’s less pleasant to reflect on our responsibilities, we should also spend some time discussing the spirit of trademark law. That is, when an individual or company establishes a reputation and good will in the market, the right to capitalize on that reputation and good will is their exclusive right.
Claims to the effect that a product is “bug for bug” compatible, or “1:1” rebuild of RHEL, or as the Rocky Linux site currently claims, “Rocky Linux rebuilds sources directly from RHEL” are an explicit claim that their derived product originates from Red Hat. These claims are not just misleading (rebuilds do not have RHEL’s certifications, they don’t have RHEL’s support, and they don’t mirror RHEL’s release model), they violate the essential spirit (and, I believe, the letter) of trademark protection even to the limited extent that they are true.
It is true that trademark law does not prevent all use of a trademark. It is likely that some statements about compatibility are defensible as fair use. But when the trademark is used to literally indicate that the source of the product is Red Hat, this becomes a misleading claim, and is obviously an attempt to capitalize on the reputation and good will that Red Hat has established.
Trademark guidelines may require not only that the product be renamed, they may require that trademarks not appear in the derived product or its documentation. Finding and removing those trademarks can be cumbersome and time consuming. They may be seen as “make-work”. They are, regardless, legitimate rights in the eyes of the law, and the Free Software Foundation, and the Open Source Initiative, and they should be respected by anyone who intends to develop Free Software — particularly by those who seek to criticize the practices of other developers from a moral high ground.
Concluding remarks
As is often the case, the panelists speak on Red Hat’s behalf, and they get it wrong. Panelists represent Red Hat’s position, to the effect of “we’re compliant because everything is in Stream.”
Stream is not provided as a compliance action, it is provided to foster cooperation and open development. Stream is a mark of Red Hat’s commitment to open source development.
Compliance and commitment are two different things, and I think it’s vital to differentiate them, because compliance is a fairly low bar, and commitment is often a very high one. Red Hat meets their compliance obligations by providing source code to the customers to whom they provide executable software. Compliance should be enough to defend a developer from undue criticism. A low bar for the vendor’s responsibilities merits a low bar of the community’s responsibilities.
But Stream is above and beyond mere compliance. Stream is open in a way that CentOS never was. It improves the process by providing a single continuously supported release branch. Red Hat increased their support for the project by accepting bug reports for Stream, which they did not historically do for CentOS without first replicating problems on RHEL. It allows developers and customers to directly suggest changes to improve or fix the product, giving them more control of their own destinies than they’ve ever had. That’s praiseworthy.
Special mention to Matt Wilson
At around 19:30, when the panel allows questions from the audience, Matt Wilson asked an insightful question that I think deserves special mention. He asked, “Is it a GPL violation to void a warranty if someone modifies software,” and further clarified, “In my mind, a warranty is a future promise to fix a defect in software or fix a device. So, it’s providing a service in the future. It’s a service agreement for future performance of services that’s voided by an exercising of software freedom.”
The allusion seemed to be lost on the panel, but I think it’s clear that Matt was drawing a parallel between the practice of voiding warranties when an essential Free Software right is exercised (the right to modify software) and the practice of discontinuing a support agreement when an essential Free Software right is exercised (the right to re-distribute software).
I agree with Matthew that these two situations are comparable, and I’m certainly curious whether the representatives of derived distributions are as zealous in their criticism of voided warranties when software is modified as they are of Red Hat’s subscription agreement.
Good question, Matt!