Web. Period.

I have read with great interest the last blog article published, « with W3C team’s hat down », by Ivan Herman on the W3C Blog. In short, he regrets the Web has forgotten about “documents”, explains what is an EPUB, and why it’s « impossible to “just” unpack an EPUB file onto a Web site and expects the result to be “just” enjoyable » and ends with a call to Web experts and ebook experts to « work together » in the recent Publishing@W3C activity, started after the merger between W3C and the ebook Consortium, the IDPF.

This made me raise eyebrows. Enough to feel the need to reply.

First, I disagree the Web has forgotten about “documents”. Yes, the Web has new (well, not that new) classes of users, like apps, and they require a lot of new stuff to make them happen and spread. But apps are documents themselves and prose-based documents are still first-class components of the Web:

• when you reach knowledge bases for a product you have purchased, it’s documents
• every time you’re on Wikipedia (and you do that quite often, don’t you?), you’re reading documents
• every time you’re reading a review for a product on Amazon, you’re reading a document made of multiple chunks of prose aggregated into one single document instance
• every time you read something on Medium, it’s a document, right?
• every time you read about the last geeky piece of news on CNet or The Register, it’s a document
• when you read online Le Monde, the New York Times or your favorite local newspaper, through the front-end Web or through an app, it’s a document
• etc.

Our Web, the one you or I use every day, is full of documents, a thousand times more than twenty years ago. I have literally witnessed the explosion of documents on the Web.

The only supposedly Web-based “documents” that are almost not readable using a Web browser (if you except with Edge that has a builtin EPUB reader or using an extension to Firefox or Chrome) are EPUB ebooks.

Let me explain…

First, a simple technical blocker : the various tables of contents contained in an EPUB are all contained in a XML file (the OPF file) that is a bit convoluted, to say the least. One of the main tables of contents (there can be several ToCs in EPUB, but let’s not dive into that complex issue…), the spine, is not linking directly to the various documents that compose the ebook… They contain a link to another link, through a mechanism known as ID/IDREF, and that second link targets the document composing the EPUB. That alone makes Web browsers choke. And even if Web browsers were allowing to dereference link targers, they would still need to implement XML anyway. Which some legacy EPUB Reading Systems do NOT implement…

Second, any normal Web engineer would assume that the “EPUB root” document — the document to rule them all — sits in the root directory of the unzipped EPUB package. Nope my friend, that would be too simple… To find the root document (let me repeat: it’s a XML document in a very specific dialect), you first have to load another XML document, called the container.xml file. That file itself is not in the root directory either, it's inside a subdir called (mandatory) META-INF. From there, you can find multiple renditions of your EPUB packages so you have to select one, based on your Reading System capabilities (the whole system of rendition selection is completely flawed by design but that's another issue for another post). So let's suppose your EPUB has only one rendition (the wider case) and you then have a URL to your XML file containing all the Tables of Contents and the metadata. Let's even suppose your Web browser can render XML, all modern browsers can do it. Your file is still not really viewable: first, you don't have any default stylesheet for your container.xml file and the rendered view of the document is... well... ugly enough to make an average user ask for a refund. Second, the URL to the OPF is inside an attribute called full-path. It’s called "full-path" because it's relative to the root of the unzipped EPUB, don't even ask me to comment on that counter-productive choice when all other URLs in an EPUB are relative... But how do you want a regular Web browser to know that the contents of that full-path attribute is a URL to a rendition and should be presented as a link? Just for the record, this is how a XML-capable browser, Firefox Quantum, shows container.xml and there is nothing clickable there...

container.xml file rendered by Firefox Quantum

That alone implies a Web browser cannot render an EPUB without a programmatic layer. And honestly, you (as a ebook reader) probably don’t want to be prompted about multiple renditions, you just want to read the ebook you purchased.

Third, all the other things that make the Web the Web are a problem in the ebook world:

• there are legions of ebook reading systems that were never updated and live years behind the current state of art of Web browsers
• EPUB is far from being the only format in the ebook world
• html, CSS are almost never up to date
• JavaScript is rarely allowed
• the Web is backwards-compatible: I can still open a mcom.com page from the Web Archive and read it, I can still open one of the very first Web pages authored by Tim Berners-Lee some 27 yars ago and read it in a browser released yesterday. The world of EPUB is not: EPUB 3.1 is incompatible with 3.0.1 that is itself incompatible with 2.0.1. Every generation completely breaks compatibility backwards and forwards.
• in return, because a market is a market, Reading Systems’ vendors rarely implement the conformance criteria of the EPUB specifications. Because otherwise they would choke on both older and newer versions.
• and of course, because a market is a market and because the versions are so incompatible, publishers keep old EPUB 2 books forever… The upgrade cost is prohibitive anyway for both the documents and the software chains handling them when there are millions (yes, millions) of ebooks to upgrade.
• while the Web is both HTML (more and more) and XHTML (less and less) without issue, EPUB is XHTML. When the whole world eventually moved to html5, the EPUB world was still struggling with XHTML 1.1 (no kidding). The HTML serialization of html is still an unknown alien in the EPUB world.
• and so on.

The ebook world is complex because most of the original technical choices were made there without input from the people of the Web (namely the W3C), because EPUB appeared and grew in a silo, and the only reason why you cannot « “just” unpack an EPUB file onto a Web site and expects the result to be “just” enjoyable » is EPUB itself and certainly not because the Web has forgotten about documents. If the Web lacks some technical features for instance on the CSS or DOM sides to render well ebooks, it’s mostly because the EPUB world was not at W3C to ask for them.

Seen from here, EPUB is a technical dead end. The ebook market just cannot absorb newer versions of EPUB any more, and I’m not sure when it will be able to absorb even light incremental changes again. EPUB books based on EPUB 3.0.1 or a light and for once backwards-compatible evolution of 3.0.1, are here to stay for a very, very long time.

The convergence with the Web Ivan is calling for has a name: it’s a disruption. The only good way to leave such a dead end that is not Web-friendly is to rethink ebooks from the ground up, based strictly on Web technologies and with the browser vendors in the loop. The Next Generation of ebooks will probably rely only on html, CSS, SVG, images and all the regular stuff you find on the Web. Probably not XML and JSON-based manifests because that’s another dead end. Conversion from EPUB to that new format will be automatable and even doable on the fly for book sellers. And browser vendors will need publishers in the loop to bring their feature requests and publishing expertise to the html and CSS standardization Groups instead of having two isolated siloes.

In both the introduction and the conclusion to his article, Ivan says we must continue « developing a Web Operating System ». I cannot disagree since I don’t even understand what he means there. There is one single Web, and the walls between apps and documents have fallen long time ago. There is no « balance between a Web of Applications and a Web of Documents as two, complementary faces of the World Wide Web » because it’s the same thing, exactly the same thing. All systems that make a difference are doomed to fail. It’s time for the ebook world to finally understand it and embrace the only Web we have and let the proprietary languages, XML dialects and isolated technical choices, die to make ebooks live inside the browser just like the rest of the Web. All the rest is futile.

Daniel Glazman