NuNet Technical Roadmap Update Q1
Greetings NuNetopians,
At NuNet, we are committed to pioneering the future of decentralized computing, ensuring that our community is informed and engaged every step of the way. In this quarterly update, we delve into the milestones achieved, challenges encountered, and the exciting path that lies ahead in our technical roadmap. Join us as we explore the progress and innovations that continue to shape our journey towards a decentralized future.
DMS Refactoring
Identification/Addressing System (DIDs)
We have been thinking about designing and implementing fundamentals of pseudonymous identity management and authentication system fundamentals, and DMS 0.5.x. refactoring milestone is a proper moment to start doing that. We are set to implement pseudonymous identification of each machine and component system (possibly using DID standard, but not necessarily).
This layer will put fundamentals needed for further technical aspects of the platform, some of which are to be implemented in short term and others in medium term: telemetry data collection and analysis, accounting system and eventually — network tokenomics design, reputation systems, etc — where ability to identify constituents involved in a transaction is an essential requirement.
IP over LibP2P Layer and interface
The IP Over Libp2p layer is basically a VPN on top of the peer-to-peer network that already exists. We’re planning to use this network for applications that need the regular ipv4 addressing such as Kubernetes and also the Decentralized SPO use-case. We did a few tests since last week with two DMSs that have established a private network between them using this functionality where one is running a Cardano relay node and another running a Block Producer node. The nodes were able to successfully sync. The main focus for this week is to make the implementation production-ready and start the design of the new network package alongside it.
Executors
One goal of the refactoring is to make the code base as easily maintainable and extendable as possible. In order to achieve that, we have started switching to using modules that are generic and extendable. Previously the DMS supported Docker for containers and Firecracker for VMs but the implementation was too tightly coupled with the use cases. Now, new modules have been added that allow support for additional execution environments with minimal code change to the rest of the application. Soon plugins will be supported to include more functionality from the community without having to change the DMS code base.
Private Networks/Channels
The private network feature enables users to create and join private networks atop NuNet nodes. This functionality empowers users to create secure, private networks with authorized peers, facilitating various use cases that necessitate knowledge of involved peers in transactions, computing, and more. Here are some technical notes about this feature:
- It is required that all peers have onboarded to the NuNet network and are using the same NuNet channel since the identification of peers is done using libp2p public key generated during the onboarding process.
- These peers need to be identified beforehand to use this feature.
- This Readme file contains links to the specification of this feature and basically describes the available functionalities to createPrivateNetwork, configurePrivateNetwork, exchangeSwarmKey, joinPrivateNetwork, rotateSwarmKey and disconnectPrivateNetwork.
Data Model/ Specification /Documentation System
To ensure the development of a functional and efficient platform, it is crucial to specify its features to meet current requirements while allowing for future needs. To achieve this, we have established a process in NuNet for specifying new features, accessible via the following link.
Following the implementation of proposed features, the specification is updated to reflect any changes made during implementation. This updated information becomes part of the system documentation.
Following this process, a feature is specified by:
- Creating a data model to represent the data structures necessary for its functionalities.
- Creating sequence diagrams to show the functionalities and interactions between the actors.
- Creating Gherkin feature files to describe the functionalities using natural language.
Test Management System, CI/CD, SecOps, etc
The combined integration of Test Management Systems (TMS), Continuous Integration/Continuous Deployment (CI/CD), and Security Operations (SecOps) represents a fundamental shift in modern software development, emphasizing efficiency, quality, and security throughout the entire development lifecycle. We are developing in NuNet:
- A robust TMS to ensure organized and traceable testing processes, enhancing collaboration and facilitating informed decision-making.
- CI/CD pipelines that automate build, test, and deployment tasks, accelerating delivery cycles while maintaining software quality through automated testing.
- A SecOps that embeds security practices into development workflows, enabling early detection and mitigation of vulnerabilities.
Public Alpha Solutions
Platform Off-Chain Security
The Security Team is working on running SAST automation and procedure on NuNet Code repositories which will increase vulnerability visibility and mitigate risks. Also, this process will be a part of the reporting QA tool and able to quality control in one place.
The team is actively expanding its horizons with the implementation of a state-of-the-art Vault solution, marking a significant stride in our commitment to safeguarding sensitive information and credentials. This initiative is in line with our ongoing efforts to enhance Test Management, CI/CD pipelines, and SecOps practices, ensuring a holistic approach to security and operational efficiency.
Job and Machine Benchmarking
The main functionality of NuNet platform is the ability to find the best machines able to execute jobs that are required by the users. The job orchestration logic and protocol will be able to deploy jobs on the proper machines. Benchmarking refers to the process of detecting computational capabilities of machines (NuNet nodes) and declaring them so that the network can access that information. On the jobs side, this means declaring the requirements for hardware capacity that running those jobs will require.
Interface to mainstream Frameworks (e.g kubernetes)
NuNet aims to provide an interface for established serverless orchestration frameworks (first of all Kubernetes, but also others) to leverage compute resources provisioned via NuNet platform. Conceptually, this means interface between centralized deployment and compute orchestration philosophies and decentralized technologies that we leverage at NuNet. Within current development efforts we start with Kubernetes integration layer, which involves a few general aspects (like ability to connect to NuNet Nodes on peer-to-peer network using IPv#4, etc.).
Furthermore, and not less importantly, we are specifying and designing additional components for seamless access to NuNet provisioned machines as pods in Kubernetes logic. This is an exciting development avenue which relates to the interface design and implementation of many other aspects of NuNet platform — job orchestration, networking, etc. The larger goal of Kubernetes integration is to define those general interfaces and their implementations on NuNet side which will allow us to easily integrate in other mainstream frameworks (e.g. for ML/LLM/SLM like Horovod.ai, Colossal-AI or learning-at-home/hivemind among many others).
Monitoring/Telemetry/Accounting System
It goes without saying that a design of a proper decentralized, or even a centralized but micro-service based (serverless or not) system, a proper telemetry and analysis system, is a must. At NuNet, we are designing a single system for collecting telemetry information (actually we are a bit abusing the “tele” part of the concept, since we are including locally collected information into the interface definitions) and exposing it via APIs to further internal and external usage. Internal usage is for debugging the network performance and functionalities of the platform; external usage of the telemetry and monitoring information would be exposed via developer tooling that will enable service providers to debug concrete applications. Furthermore, the accounting system will be based on part of the telemetry information which will enable all constituents of the platform to access their own usage and deployment data, which possibly is private to them.
Decentralized SPO Use Case
At NuNet, we embarked on a mission to empower Stake Pool Operators (SPOs) by leveraging community-provisioned hardware, with the ultimate goal of enhancing the resilience and decentralization of the Cardano Network. Our vision was grounded in implementing secure and efficient functionalities for the Cardano ecosystem.
In the early months of 2024, our journey has been both challenging and enlightening. We’ve made considerable strides towards creating a decentralized ecosystem for compute providers and SPOs. One of our key focus areas has been the security of our platform. To this end, we explored using Firecracker, a very lightweight virtual machine (microVM), to enhance the security of operating environments for compute devices. This technology, while promising, presented unique challenges, especially in ensuring both internal and external security measures to protect against potential threats.
Our infrastructure developments have seen significant progress, particularly with the successful provisioning of hardware that establishes the foundation for our platform’s scalability. However, the implementation of a reward system that adequately compensates providers and SPOs is still underway. This has underscored the importance of refining our hybrid model to ensure it aligns with our vision of creating a fair, sustainable ecosystem for all participants.
We’ve also made pivotal adjustments to our strategic milestones based on recent insights and challenges encountered. For instance, the pursuit of a completely secure operational environment led us to reevaluate our strategy for using Firecracker VMs. Instead of encrypting the Firecracker filesystem, which proved impractical, we are now focusing on strategies that avoid storing sensitive information within the VM altogether. Additionally, our commitment to enhancing the usability of our platform is exemplified by our ongoing development of secure shell access, which, though still in progress, marks a significant step towards providing a more secure and user-friendly environment for node management.
Looking ahead, we’ve refined our deliverables to better align with our objectives and the evolving landscape. This includes efforts to streamline the onboarding process for compute providers and to set up private relay nodes that offer SPOs enhanced connectivity and security. Our exploration into innovative solutions for blockchain data storage and access continues, as we remain open to pioneering technologies that could offer greater flexibility and security for blockchain data.
As we continue on this path, our resolve only strengthens. Each challenge we face and milestone we achieve brings us closer to realizing our vision of a decentralized, secure, and efficient Cardano ecosystem. We’re committed to transparency and innovation, and we look forward to sharing more updates as we progress. Join us on this exciting journey towards revolutionizing decentralized computing.
NuNet Is Hiring!
NuNet currently has a number of open positions for various roles within the team. If you have the skills and desire to join us in our journey, you can find more information and contact us through our career page.
About NuNet
NuNet lets anyone share and monetize their computing resources, turning cloud computing power from a centralized service into an open protocol powered by blockchain. Find out more via:
