Xanadu vs. the World Wide Web: a success/failure story
Project Xanadu was a widely-touted attempt to create a global, networked Hypertext publishing system. It was founded in 1960, three decades before the web was introduced in 1989. It shared many of the same ideals and aspirations. Yet, despite a substantial engineering team and corporate sponsorship, it never produced a viable product.
As someone interested in failure and its causes, I often think about why this is.
Part of Xanadu’s problem was that it was much more ambitious than the web. It was designed to have micropayments built in from the start: the act of reading a page would automatically send a small fee to the author of that page, as well as the authors of any embedded (“transcluded”) content in that page. Versioning and commenting were also an intrinsic part of the design. And there would be no such thing as “broken links”, since all links were bi-directional, and would automatically be updated as content was moved or changed.
Had the Xanadu team been willing to scope down their ambitions, I have no doubt that they could have put out a working implementation, and I suspect that it would have been at least moderately successful.
However, having met Ted Nelson and some of the other Xanadu founders, I know that they would never have been willing to do this. In fact, their opinion is that, despite it’s success, the world wide web is clearly wrong for not having these features — that the omission of things like micropayments and bidirectional links is a civilization-level blunder, a colossal mistake from which we can never fully recover.
By contrast, the architecture of the web clearly shows a spare, rigorous and disciplined design that doesn’t try to do too much (although it may seem complex today after 30 years of “improvements”). The web was successful because it didn’t try to take on too many design requirements.
The lesson I get from this is simple: technological innovation is often stymied because potential innovators care about too many things; conversely, innovation leaps forward when someone comes along for whom those concerns are unimportant.
The hard part for an experienced software engineer is to realize that the thing that you cared about so passionately, for so long, is no longer relevant.
This is particularly a lesson for people working at big companies. In a small startup, your main constraint is cost; but you can often compensate for the lack of capital with clever design. Big companies, on the other hand, have to worry about a lot of other things — liabilities, regulations, the potential for hackers and lawsuits. Because the stakes are so much higher, global mega-corporations are vulnerable in ways that shoe-string endeavors are not.
As a result, big companies have to be much more cautious, and that adds a substantial burden of overhead to the development process. Navigating the complex web of gatekeepers within a big company adds an enormous burden to engineering teams. Even with plenty of funding and a generous head count, the drag on innovation can be noticeable (especially when you consider that 5x staffing does not equal 5x productivity — ref. Brook’s Law.)
The trick to innovation is to care about the right things, and to not care about the unnecessary ones. I’m reminded of that quote from Chris Nolan’s Interstellar:
Newton’s third law — the only way humans have ever figured out of getting somewhere is to leave something behind.
