Complexity and cruft are the enemies of progress in software. As technologies and ecosystems mature, the original teams disperse to new projects or to found new companies. New people arrive as teams get larger and products lose the focus that made them initially so successful. So bad ideas, ones that the developers intended to fix or change, or that no longer serve the needs of the current context endure and thrive as “just the way things are”.

Paradigms freeze as new developers come to see the environment in which they learned as the “best” or “newest”, “latest” and no longer subject to revision or improvement. Ideas which once made sense in the context of a former time no longer fit the present, yet few challenge the assumptions which now form the baseline for entire ecosystems and ways of thinking.

It takes more work for new entrants to meet the aggregate baseline of features that all of us come to see as the minimum acceptable. So new products take a long time just to reinvent and duplicate the features they know their users will expect. Developers spend more time duplicating features from other apps and less time innovating. The relatively few success stories tend to be applications with narrow-focused functionality that do one new thing better than the alternatives. WhatsApp or Instagram are two of the better-known examples.

Global internet software, in particular, is built on a foundation of ideas and paradigms that limit our ability to move forward as the architectureal design decisions which gave rise to our current giants — Google, Apple, Facebook, Twitter, Linux et al. — are baked into the assumptions of almost everyone who codes anywhere. The success of a few large companies in building systems that scale are replicated across the industry since “Google and Facebook have the best engineers” so there’s no longer any point in lesser mortals looking for new approaches to old problems.

Further, in a networked world, the business scale advantages that acrue to the dominant players serve to entrench those players in their markets in ways that make it very difficult for any new software to take root in those markets. Competing with Facebook is more than just about building better software, it becomes of necessity, a battle to gain users at the expense of Facebook. It is the network of users itself that is the real value, not the particular implementation of application that Facebook uses to let us communicate with that network.

Without a critical mass of users you can’t compete, and the benefits that flow from a larger network are nonlinear. Software writers often refer to Metcalf’s Law as a concrete expression of this benefit, but this only captures part of the problem. The bigger, and more difficult problem, is simply that without lots of users and lots of data, many potential applications are useless. No users, no benefits. No benefits, no users.

So many ideas which would be excellent at scale can’t ever reach scale because the starting hurdles are simply too high for small teams, even well funded ones, to jump.

New paradigms come when some small group of people seek to make easy that which was once hard, or even possible that which was once not. What is hard now? Getting to scale. Competing with the existing networks that have Metcalf advantage. Building robust software that fits together with other robust software that other teams wrote. What if we made all of that easy?

We could do this starting with the data. Imagine a shared infrastructure service that was owned by the people who create that data and that can be accessed at scale by any software developers. Imagine if the data in Facebook, and the hardware and software infrastructure on which it ran, was owned by Facebook users and open for access by any developers who wanted to build on that data. And Facebook was no longer in a position to stand between user’s data and any uses of that data those users wished to permit.

This was the initial promise of the Facebook APIs, the initial promise of Twitter as infrastructure. A promise killed when it became obvious to the suits that more profit came from keeping the data private and from owning the entire “user experience”. The need for money drove decisions to make API access more and more restrictive, so third-party developers could no longer offer competing interfaces that rode atop the data that Facebook and Twitter came to see as their rightful property. This hamstrung the third-party apps business for Facebook and Twitter and it has never recovered.

Shared data infrastructure as a service would allow many small developers to join forces and build together a new open network which won’t ever shut out developers because some small group wants to monetize the users themselves. The very purpose of a shared data infrastructure service is to serve the users and therefore serve the developers who met the needs of those users, no matter when or where they arrived. More developers, more options, more potential benefits.

Unlike proprietary software companies like Facebook and Twitter, users won’t mind if thousands of new developers write software that can access their data should they choose to use that software. In fact, that’s what they’d prefer.

For this to work, the privacy and access restrictions will need to be part of the shared data infrastructure, so that users will be able to control what other users can see and the kinds of queries that other applications can run against their data. And since the users will own the data, they’ll control the ways in which others pay to access their data.

Want to show me some ads? Pay me the money I require or go somewhere else. Monetization and privacy schemes will be implemented at the level of the shared large-scale data infrastructure, not the application. So I won’t have to worry about different monetization and privacy settings for different applications running on the same data.

When users own their data and the hardware and software on which it is stored and accessed, they will control their futures and innovation will flourish.