The Successful Conclusion of the Crust Tests
If you’ve followed the SAFE Network project for a little while, you’ve probably heard of something called Crust. It stands for Connections in Rust, a low-level networking library that is optimised for P2P connections between nodes. In other words, it’s the way that computers connect to each other whilst building a decentralised Network.
After a couple of public tests, we wanted to say thanks to everyone who took part — whether in the initial test on October 11th or the more extensive test that we released a couple of weeks later.
Read on to learn more about why this test was valuable, and what the next steps are.
tl;dr we got the results we were hoping for! :)
Summary of Results
From the start, the level of engagement around the world was humbling. We saw over 20,000 attempted connections during the tests, with computers from 37 countries taking part, including:-
- Argentina, Australia, Austria, Belgium, Brazil, Bulgaria, Canada, China, Croatia, Czechia, Finland, France, Germany, Hong Kong, India, Indonesia, Ireland, Malta, Mexico, Netherlands, New Zealand, Norway, Poland, Republic of Korea, Republic of Lithuania, Romania, Russia, Slovakia, South Africa, Spain, Sweden, Switzerland, Taiwan, Turkey, Ukraine, United Kingdom, United States.
In the first test, we focused on testing the effectiveness of UDP and TCP hole punching by itself. To put it another way: if computers aren’t able to speak directly to each other, how good is the software at punching through the barriers that make each one invisible to the other (for example, where a computer has no public IP address as it lives behind a home router). The answer? We saw successful connections punched through (using UDP and/or TCP hole punching) around 48% of the time.
A couple of weeks later, we released a test with a significantly more effective version of Crust. The software now also enabled nodes to connect directly (in addition to using TCP and UDP hole punching), whilst we also enhanced the dashboard performance and filtering options that helped illustrate what was going on.
And the results second time round? Even more compelling: with 11,584 attempted connections, we saw a 75% success rate now that nodes had this extra option of directly connecting to other computers around the world.
What Goals Did We Have at the Start of the Test?
As the foundational communication method within the SAFE Network at launch, we’ve been working on Crust for a number of years. The goal of the Crust Tests was therefore pretty simple: deliver this milestone in order to confirm that the technology is working as designed in the wild. Furthermore, as a standalone library that’s free to be picked up for use by other projects, it was important to stress-test the capabilities at this point in time on the road to the next major inflection point — the release of SAFE-Fleming.
After all, the SAFE Network isn’t simply some kind of VPN-’replacement’ internet…
What Else Did We Find Out?
With the SAFE Network, every single piece of user-submitted data must be encrypted before it hits the Network. The SAFE Network Fundamentals require full encryption and we saw very clear evidence of why this is required in the first test. You can read more about the issues that we saw in this post — but to quote that post:
“If an ISP can change your IP address, what else is it doing to your data behind the scenes? Encryption prevents this. That’s exactly why we’ve built the Crust library with this principle in mind. It’s all very well creating a software library that facilitates peer-to-peer communication. But if (as many high-profile projects are doing today) you’re not encrypting every single piece of user data as it hits the wire? You’re little more than a Band-Aid that falls off as soon as the weather turns. And you’re no longer solving the very real problems out there.”
Again, this isn’t about getting something out there into the wild and letting it fend for itself. It’s about providing effective tools that ensure the privacy and security of SAFE users above all else. And there’s a world of difference between the two.
What Do We Still Need to Do with Crust?
Future work within Crust will focus on:
- expanding the ways in which Crust can communicate (the addition of multi-protocol support);
- the release of a more decentralised method by which a new node can dynamically join the Network (Bootstrap Cache); and
- work to ensure that all data is transported in an encrypted format as it traverses the Network (Secure Serialisation).
What Do We Still Need to Do for SAFE-Fleming?
With the work that’s been carried out to date, we now have all the information that we need on Crust in order to get us to SAFE-Fleming.
The Fleming Release is a major milestone for the SAFE Network. At that stage, we’ll be releasing a truly decentralised global routing network for transient data. We now have the evidence that Crust is working as planned. In fact, the tech that we’ve been testing in this test is more advanced than the version of Crust that is required for Fleming. SAFE-Fleming itself only requires Crust to provide the ability for computers to connect via direct connections or through what is known as UPnP connections (where an individual carries out manual port-forwarding using his or her router in order to connect to the Network). Once Fleming is out in the wild and in daily use, we’ll be moving on to including the ability for Crust to function using hole-punching also.
The next few weeks will see the release of many of the remaining constituent parts of the Routing layer (aka ‘the brains’) of the SAFE Network: dynamic membership, malice detection and sharding amongst other things. Expect to hear much more about each of these as we release them in stages! But in the meantime, thanks to everyone who took part in the Crust Tests. Each of you has just helped to move that vision of a new truly secure and free decentralised internet that little step closer.
To learn more, check out the SAFE Crossroads Podcast Episode 45 for the background to the Crust Tests and Episode 46 for feedback on the results.
