The Ants Are Coming!

It is fair to say that MaidSafe have been fairly quiet in our communication of late, not a complete radio blackout, but certainly low key. This has been intentional on our part. Communication can lose its meaning and impact if you shout about every little thing. We like to keep our powder dry, as they say.

But, now we have something meaningful to convey!

I want to bring everyone who is interested up to speed with what we have been doing during 2015, specifically the past three months, which have been some of the most transformational in MaidSafe’s nine-year history. We do make weekly dev updates on the community-run forum, but these can be pretty technical, and not everyone can get a full picture that way. So I will try and condense these points into as limited an amount of technical language as I can muster.

Rather than starting in early 2015, however, we will start this post at the present day. (I’m a sucker for Quentin Tarantino films and it will hopefully make for a less laborious post!)

Early(ish) on Friday (26th June) morning, the tireless MaidSafe dev team managed to get the SAFE Network functioning from end to end. This means that a User is able to self-authenticate (create their own username and password, and access the network without the permission of a third party) using the Authentication API via the SAFE Client. During the process, the network’s transport/connection layer (Crust) connects the peer nodes, allowing Routing (the layer that verifies the identity of each node cryptographically) to establish and maintain connections to other network nodes (for more information on how all the components fit together, please visit our wiki). The clients were also able to PUT (store) and GET (retrieve) data.

This is a phenomenal and long-sought achievement for everyone at MaidSafe, and is truly a major milestone ticked off. Yes, we have had an operational network before, but never one so stable, efficient or lacking in complexity. This lack of complexity is a very good thing and something of a personal mission for David. A significant reduction in the lines of code (from hundreds of thousands to a few tens of thousands!) not only enables new network features to be added much more quickly, it also makes it easier for bugs to be spotted and for other projects to utilise our code, much in the same way that we have employed the code of others. Open source collaboration is truly a wonderful thing to be a part of!

This new chapter in our development progress all started back in early February when David, again desperate to reduce complexity, spotted a pattern in the code, a repeatable and describable pattern that kickstarted the simplification of the decision-making logic within the Vaults (the series of processes that help look after all data on the network) and also improved network security. You can read David’s detailed blog post about it here.

Soon after, heavily frustrated by the speed at which we were developing in C++, we started looking into ways to speed up development without a reduction in code quality, surely the holy grail of software development. After much research, David became increasingly convinced that a new systems-level language, Rust, had something to offer. In his spare time (between about 2 a.m. and 5 a.m.) he started transposing one of our most complex libraries, Self-Encryption (the component that seamlessly splits data into smaller chunks and encrypts them), over to Rust, which at that time wasn’t even in Beta yet. This was very successful and fast! David followed a similar process with MaidSafe’s Routing library. With another successful test complete our development team was split for a few weeks while the core team remained in C++ and another team started transposing the rest of the code.

This was a risky and scary time. To split the dev team at a period when we were under significant pressure to produce a stable network seemed counterintuitive. But thankfully, going backwards to move forward paid off, and without this change there is no doubt in our minds that we would not be where we are today. It is not the intention to go into detail here about how and whether Rust is better than C++. For debates on that subject you can check out some of the threads on the forum and elsewhere. I think that is a debate as contentious as GPL vs MIT, or even Borg vs MacEnroe, fun to debate but don’t expect consensus any time soon. All I can say is that it’s working for us and allows us to iterate quickly, be more defined with our tasks and be more definitive with our timescales.

During this time we also realised that we needed to be more third-party-developer focussed. And that doesn’t mean being more open with our development process; we’re pretty open already. Rather, it means organising our libraries in such as way that they can be picked up and used by any third-party developer without being heavily coupled to other libraries. In addition to whatever assistance this gives to other projects in the open-source community, this helps Project SAFE in two specific ways: first, allowing external use of these discrete libraries helps validate their functionality in a variety of different projects; and, second, it allows us to demonstrate, at a library level, that each component of the network carries out its desired function.

So, beginning in late April we started to release each of the component libraries as console apps, essentially providing the libraries functionality with a text-only interface (such as terminal in mac and linux, or the command line on Windows) rather than a Graphical User Interface. The first console app was Self-Encryption, and since that time we have also released Crust and Routing. All MaidSafe libraries are published on crates.io, where a stable release of each library can be used by other projects, and dependencies known (dependencies are the other programs that the library relies on to run). This process is not only useful for third-party developers, it also gets the MaidSafe team into a habit of delivering working code regularly and in a routine way.

We have found that this more modular and more approachable way of working has led to many more developers committing code to the repositories, and it is valuable code. Such is the desire for this network, and also the desire of people to try Rust. We will accommodate these desires and pay anyone who contributes to our code sprints, starting from this upcoming sprint (we will update everyone on the bounty scheme next week on this blog). We anticipate this will give us better code faster and with a more engaged and willing community, the impact of this cannot be underestimated. This is an area we will work hard on, to ensure the community is well-rewarded and recognised for contributions, as we feel they should be.

So, now that we have a running and more stable network than we have ever had before, what next? What does the future hold? Well, we can look to our roadmap for the answers. In the very near future (in the next couple of weeks), we will be releasing installers to enable users to download and run a SAFE Network locally on their individual computers. Beyond that our next development sprint will start week commencing July 6th (probably lasting for around three weeks) and though we are still mapping out specific objectives, this sprint is likely to consist of some or all of the following:

• Further implementation of safecoin (some work here has already started)
• Implementation of messaging infrastructure
• Remove Transaction Managers (reduce complexity, code and increase security on the network)
• Implement App Launcher infrastructure (enables secure login to third-party apps without the App Developer getting visibility of a User’s credentials)
• Implement cross-platform/cross-network/multi-protocol network connections (everyone can run farming Vaults at home or in any network)
• Implement POSIX disk interface (any app can treat the network as a local hard disk)
• Implement public names and shares ( SAFE version of www, email, DNS and more)

From there we should quickly (hopefully within 2 sprints) see the delivery of Dev Bundle 2. This iteration will provide a network where Farmers can start to contribute resource and have this measured, where developers will be able to access and utilise stable APIs and start to build applications that everyone can use. These apps might be simple at first, such as encrypted messaging and data storage apps, but in a short space of time will have the potential to develop into anything that can be done on the existing web and beyond. We will possibly split out Dev Bundle 2 on the roadmap so that some of what is currently listed will become Dev Bundle 3. This way the deliverable of each feature focussed sprint will be an updated bundle.

So, this recent development milestone is hugely significant and we are very close to delivering the SAFE Network, something that the world would seem to need even more than it did when David started this journey nine years ago. We will continue to keep you all updated via the forum and this blog.

Thanks all, for your continued participation and support.