Decentralization in Light of the Recent Amazon S3 Downtime
Today (February 28th, 2017), Amazon S3 is experiencing a significant outage. Multiple regions are out, and as a result a significant portion of the Internet has become unusable.
Ironically, we released v1.1.1 of Sia on the same day as the outage, and people found themselves unable to download our new release as a result. We were able to publish the release through alternate means, though the ‘Update’ button inside of our software is not working (the software itself still works fine, you just needed to be creative if you wanted to grab the most recent update).
As of writing, we do not know what caused the outage. But we know that many regions have been affected and that the outage has been limited to a single centralized service. My money is on a faulty config file being pushed somewhere. Or maybe Amazon’s version of the Chaos Monkey got out of hand.
It is bold to say this without knowing exactly what caused the outage, but decentralized systems are immune to this type of downtime. And that’s because there is no central system, no single config push, no single line of hardware running the system, no master node that give out faulty or malicious commands. A properly decentralized system can’t be taken out by the faulty actions of a single person, because it will have explicitly been designed around assuming that any minority of people (including the network designers and developers) may suddenly attack the network with their full strength and creativity at any time.
And indeed, Sia operates this way. Today we have about 85 hosts online running a mix of Windows, OS X, many flavors of Linux, and even FreeBSD. Our hosts are on 4 continents, are running 6 different releases (v1.0.0, v1.0.1, v1.0.3, v1.0.4, v1.1.0, v1.1.1), all behind completely different networking setups. This is all coordinated over a blockchain, which itself is very difficult to take down, but even if you managed to take out the blockchain for a day or two, the nodes would continue allowing uploads and downloads through their existing contracts (you just wouldn’t be able to form new contracts until the blockchain was working again — but again the blockchain is really not the weakest link here).
Everyone using the Sia network has spread their failure across a large set of these nodes. People still running v1.0.0 are using a 4-of-24 redundancy scheme for their data. People running v1.1.1 are using a 10-of-30 redundancy scheme for their data. People running custom software could be using a 3-of-60 redundancy scheme. Each person has a different view of the network, meaning each person is forming contracts with different hosts. A wide outage is really only fathomable under a carefully planned attack. And even then, many of our hosts are running out of datacenters with gigabit and even ten-gigabit connections.
And these are the very early days. Today there are 85 hosts, but we expect that in 12 months we will have several hundred. And we also expect that the average host quality will be steadily increasing with time, even as the host count grows. The network has been explicitly designed to encourage competition and to reward the most competitive.
The premise of a decentralized network is that there are no single points of failure. And, ‘single point’ is underselling the concept. Really, the goal is that any of dozens of things could go wrong simultaneously without causing any noticeable slowdowns or outages, and without putting anyone’s data or money at risk.
Bitcoin has achieved this with money. There have been several heavy attacks against the Bitcoin network, and yet it has continued to operate every day without downtime. When the throughput limits are hit, the transactions that get served are only the ones willing to pay for it, but anyone who is willing to pay for it (price has yet to break $0.50 per transaction) has been able to use Bitcoin.
Sia is doing the same thing with data storage. Today the network is young, and the software is still maturing, however in full maturity Sia will be a complete replacement to S3. Capable of greater scales, at better prices, with lower latency, higher throughput, and stronger security, all without any single point of failure, including the failure of the parent company and the developers. Outages like today’s S3 downtime and vulnerabilities like last week’s CloudBleed will be behind us.
Today’s Internet infrastructure is in desperate need of decentralization. And while it’s needed in more places than just data storage (we could really use decentralized email, search, social networks, and news aggregation as well), we’re happy to be doing our part to make the Internet more reliable and more resilient.
