NuNet Technical Update Q2 2024

Published in

NuNet

12 min read3 days ago

Greetings NuNetopians,

As we step into the third quarter of 2024, we want to address the delays in delivering our roadmap promises set up in the beginning of the year. As an open source project we believe in transparency and feel it is important to communicate openly about the challenges we have faced and the steps we are and have taken to overcome them.

Our 2024 Technical Roadmap update (post) set the following timelines for the milestones that we are actively working on:

DMS Architecture Update and Refactoring (a.k.a. Device Management Service Version 0.5.x milestone) — second half of Q2
Public Alpha Solutions — end of Q2
Public Alpha Mainnet — start of Q1 2025
Decentralized SPO Computing — second half of Q2
Public Beta Mainnet — 2025, but further undefined

Design and implementation of all these milestones crucially depend on the Device Management Service and thus on the release of the Device Management Service Version 0.5.x milestone. We have made considerable and very tangible progress towards releasing the DMS 0.5.x version (see details below), but we are late, which pushes back implementation of most of the other milestones and use-cases. As of current estimations, we are set to start release process (i.e. locking DMS feature scope, scheduling release candidate builds, community testing using the custom development tooling) first half of September 2024 and aim to complete the process with final release of DMS 0.5.x at the end of the year.

We will communicate the full release process description and roadmap in a separate blog in the beginning of August. In this blog post we want to address the general update to our roadmap in the light of the learnings from the Q1 and Q2 of 2024.

Challenges and Learnings

Major learning from pushing the 2024 Technical Roadmap is the intricate relationships between all milestones and the importance of architectural and design purity of DMS 0.5.x release which is fundamental for reaching our technical and business goals in terms of all other milestones and in the long term. With that in mind, we took steps to explicitly communicate the technical concepts that we were following all along both internally in the team as well as with the community (see our blog posts series on Open Ended Computing concepts). While the ‘DMS Architecture Update And Refactoring’ stage was planned with precisely this importance in mind, reaching it took us longer than expected.

Another major learning (which is also a challenge when following it) is the realization that we need and can move forward in parallel with most of the milestones while updating the architecture of the platform by leaning heavily on specification and documentation system (which has been developed for more than a year and is included into the release process) and clear interfaces and their implementations. For example, ip over libp2p and Kubernetes integration are included into NuNet Solutions milestone, but are being developed together with interfaces that are backed deeply into the DMS architecture. Similarly, the research and development towards training and deploying Small Language Models on decentralized hardware infrastructure is showing good progress (see technical discussion) albeit will be offered to users only after DMS 0.5.x release.

Our overarching goal that we are striving for with the release of DMS 0.5.x is offering our partners and community participants a robust but evolvable software architecture, code, contribution process and developer tooling that will enable us all to bootstrap the ecosystem of open-ended decentralized computing that would hold the potential to change the way we use the ever increasing capabilities and requirements for access compute resources.

Device Management Service Version 0.5.x

The term DMS refers to ‘Device Management Service’, which is essentially the NuNet platform itself. It is the lightweight binary which establishes secure peer-to-peer connection between all computers on boarded onto the network. Work began at the start of the year to refactor the DMS with the goal of implementing our Open-ended Computing model architecture that would create a more streamlined system architecture, enhance scalability and maintainability, and better integrate emerging technologies.

Network

We are pleased to provide an in-depth update on the enhancements to the network package within our DMS. Understanding the vital role of the network package is important to understand the advancements we have made and how they contribute to the overall platform.

The network package in NuNets DMS is a core component designed to manage and facilitate communication between various devices and services within our decentralized network. It serves as an abstraction layer, enabling seamless interaction and data exchange regardless of underlying network protocols (such as libp2p). This package supports different types of messaging which is crucial for maintaining efficient and reliable operations across the network. This package is now merged into NuNets DMS code:

Key features and Enhancements

Abstraction of underlying network libraries:
During the past quarter, we focused on abstracting the specifics of the underlying network libraries. The package provides a generic network functionality, which results in simplifying future integrations with other libraries such as NATS. This abstraction layer increases scalability and flexibility of our platform.
Support for connectionless and connection-oriented messaging:
We have merged this code which allows the platform to cater to different application needs, the network package now supports connectionless and connection-oriented messaging. This is needed for the actor model which avoids keeping connections open for sending messages. However, for functionalities such as file transfer or log streaming, it will be necessary to keep a connection open.
Address-Handler pair registration
We have implemented a feature that allows the registration of address-handler pairs. This enables messages to be routed to specific handlers at the application level which enhances efficiency and modularity allowing for more complex actions in the system.
Private Network Functionality
Another milestone met this quarter is the implementation of private network functionality. This feature allows users to create and join a secure private network with authorized peers, which enhances security and enables tailored communication channels within the NuNet ecosystem.

These enhancements to the network package are vital to improving overall efficiency, security and scalability of our platform. By abstracting network complexities and supporting different messaging needs, the network package ensures that the NuNet platform can handle various use cases and adapt to future technological advances seamlessly, in line with our open-ended computing architecture.

Executors

The executors package in NuNet’s DMS is a core component designed to manage and facilitate the execution of jobs in different job environments seamlessly, simplifies operations and enhances efficiency. This package is now also complete and merged, after which the platform supports two execution — Docker and Firecracker — and implements a flexible interface to add compatible executors (virtual machine or containerization system):

Key advancements and features

Single job execution view
The executors package now provides a single, cohesive view of job running environments. allowing the DMS to manage different execution environments seamlessly.
Flexible environment support
Designed to treat containers and VMs in a unified manner the executors package allows different instances of the DMS to have different job execution environments based on the installed capabilities. Whether using Docker or Firecracker, the DMS can manage these environments consistently.
Extendable architecture
The architecture of the executors package is very flexible. It is designed to support additional execution environments such as WebAssembly (WASM) VMs. This ensures our platform can adapt to emerging technologies and integrate new environments with minimal effort.

These advancements in the executors package greatly enhances the versatility and scalability of the platform. By providing a unified view and flexible support for various execution environments the DMS can efficiently handle diverse job requirements, ensuring robust and reliable operations.

Storage

The storage module/package, which is now complete, in NuNet’s DMS package is designed to facilitate seamless data access for running jobs. It utilizes various storage service provider protocols in a unified volume, ensuring that jobs can access data regardless of its location or underlying storage framework.

Key features and enhancements

Unified storage interface
The storage module includes a storage abstraction layer that allows the DMS to integrate with multiple types of storage mechanisms, including local disks and external storage providers like IPFS and AWS S3. This unified interface simplified data access across different storage providers.
Implementation of storage mechanisms
We have implemented a chosen set of storage mechanisms using the unified interface. This ensures that the DMS can access and manage data from various storage solutions effectively. Including local storage, docker volumes, HTTP/FTPS, Google Cloud, AWS S3 and more.
File transfer between DMS instances
The storage module enables DMS instances to transfer files between each other. This functionality relies on the network module for data transfer while the storage module handles reading, writing, error checking and seeking specific locations within files.
Disk storage management
We have developed disk storage management capabilities for DMS, ensuring efficient data handling for both the DMS itself and the jobs it deploys. This also includes adapters for external storage services.
Encryption at rest
To enhance data privacy, the storage module offers an encryption at rest option. Users are able to provide their own encryption keys instead of relying on the default system provided encryption.

The completion of the storage module/package improves data management capabilities within the NuNet platform. By providing a unified and flexible approach to integrating various storage protocols, the DMS ensures that data is readily accessible for all jobs, regardless of its storage location. The added functionality for file transfer between DMS instances and encryption at rest further enhances the security and efficiency of our decentralized network. Learn more about our storage package design and implementation on GitLab. Related merge requests:

Database

The database package in NuNets DMS is designed to facilitate data persistence, enabling efficient data retrieval and storage for various operations within the platform. The refactoring has introduced new capabilities and improved the package’s overall efficiency and versatility.

Key features and enhancements

Support for multiple database types
The refactored database package now offers support for a range of database types, including document databases. This allows the DMS to utilize the most suitable database type for different use cases enhancing data management efficiency.
Decoupled database implementation
Database implementation has been decoupled from its usage within the DMS. The change allows developers to focus only on record storage without being tied to a specific database, increasing efficiency.

These enhancements to the database package significantly improve the data management capabilities of our platform. By supporting multiple database types and decoupling the implementation from its usage, the newly refactored DMS ensures greater efficiency, flexibility and ease of maintenance. These developments allow developers to focus on building and optimizing applications without being constrained by database-specific requirements.

Telemetry

Finally, we are happy to share the recent advancements in our Device Management Service (DMS) telemetry package, designed to enhance observability and provide comprehensive insights into system performance and behavior. During the last quarter, we designed and implemented the DMS telemetry package that contains logs, tracers and everything related to telemetry.

Key feature and enhancements

Comprehensive Instrumentation
The telemetry package generates and manages traces, metrics and logs ensuring full system observability.
Flexible and extensible implementation
The package supports a wide range of observables and collections, which provides a flexible and extensible telemetry system. This allows the DMS to cater to different telemetry requirements effectively.
Enhanced observability
With the implementation of the telemetry package, developers can gain deeper insights into the system’s performance and behavior, which aids in identifying issues, optimizing performance and ensuring the reliability of the platform.

The design and implementation of the telemetry package can be found in the following README files:

Instrumented packages

Two DMS packages are already instrumented using the telemetry implementation:

Storage package
The storage package has been instrumented to generate telemetry data, providing insights into its operations.
Database package
The database package has also been instrumented to generate telemetry data, enhancing its observability.

The implementation of the telemetry package significantly enhances the observability of our platform. By ensuring comprehensive instrumentation and supporting various observables and collection the telemetry package enables better monitoring, debugging and optimization of the DMS.

For more info on NuNets telemetry package check out our recent technical discussion ‘Telemetry instrumentation’.

Orchestrator and Capability model

The orchestrator and capability model in DMS are currently defined and undergoing implementation. These components are crucial for coordinating various system functionalities and enabling efficient resource management within the network.

Orchestrator Package

The orchestrator package manages the overall coordination and operation of DMS. It handles the allocation of tasks and resources, ensuring that the right components are activated and utilized efficiently. The orchestrator’s functionality includes task scheduling, resource optimization, and system monitoring to maintain seamless operations across the network.

Capability Matching Package

The capability matching package is responsible for identifying and matching the capabilities of different devices and services within the network. It describes how capabilities are defined, matched, and utilized, facilitating the efficient pairing of tasks with appropriate resources. This package includes a class diagram to illustrate the structure and relationships involved in the matching process, ensuring a clear and organized implementation of capability management.

For further details, refer to the README files available on GitLab:

Decentralized SPO use case

The Decentralized SPO Computing milestone is a commitment from NuNet to enhance the resilience and decentralization of the Cardano Network. NuNet aims to leverage community-provisioned hardware to support Stake Pool Operators (SPOs) with secure and efficient computing resources.

Key Updates

The milestone was temporarily paused to await the implementation of the IP over libp2p functionality within the DMS network package. This functionality is crucial because it eliminates the need to develop a new transport system for SSH-like communication and a terminal emulator for the SPO.

With the IP over libp2p functionality now available for testing, as detailed in merge request #362, development efforts will focus on integrating this capability into the decentralized SPO computing framework. This integration will facilitate robust and secure communication between SPOs and their computing environments, leveraging the established SSH protocol

Using IP over libp2p for SSH access is expected to significantly enhance both the security and efficiency of the decentralized computing environment for SPOs. This approach ensures that all communications are encrypted and reliable, aligning with our commitment to providing top-tier security for decentralized applications.

Availability for Testing
The implementation of IP over libp2p is now available for testing, as detailed in merge request #362. This progress allows us to resume work on the “Decentralized SPO Computing” milestone, integrating this new functionality to provide SPOs with seamless and secure shell access.

Public Alpha Solutions

In the last quarter, we evolved in two important aspects of the Public Alpha Solutions milestone: (1) the IP layer over libp2p and (2) the integration layer for Kubernetes cluster.

IP over libp2p

Implementing an IP layer on top of the libp2p network unlocks numerous potential use cases for public alpha solutions. Over the last quarter we modified the demo implementation of the IP over libp2p from the 329-enable-distributed-jobs-based-on-vpn to a production ready state. This involved extensive optimization and security improvements discussed in this GitLab issue. Various network scenarios were considered to design the final solution, detailed in the discussion note.
The implementation is now available for future testing and can be found in this merge request.

Subnet optimization

This README file covers how the system optimizes IP subnets over the libp2p network. The architecture and implementation of subnet optimization, including communication protocols and configuration settings required for deployment. The README covers instructions and guidelines for setting up and managing subnets within the libp2p framework, ensuring efficient and reliable networking.

These developments in IP over libp2p are crucial for enhancing the functionality and scalability of NuNet’s Public Alpha Solutions. By enabling robust network configurations and efficient communication protocols, which paves the way for more secure and sophisticated decentralized applications.

Kubernetes integration

The Kubernetes integration with NuNet will allow Kuberenetes users to leverage NuNet’s compute power while maintaining their existing workflows. We have done extensive research and discussions of which the results can be found in this blog detailing the integration specification and architecture.

Key Developments

Interlink Plugin Development
We are developing a plugin for an open source tool called Interlink. Interlink is a new tool designed to connect Kubernetes with supercomputers and consists of two parts, a virtual kubelet that is deployed on a node in the kubernetes cluster. This then communicates with a backend service residing on a server outside the cluster. (We are calling this the interlink Gateway. The reason for this is that alongside the interlink backend we need an authentication system, our plugin and a locally installed DMS.)
Security Configuration
Interlink has just released Version 0.3.0 which includes several changes that will help us enable the security configuration we want to use. We will be using Keycloak to authenticate the virtual kubelets using (JWT) JSON Web Tokens with the help of an OAUTH2 proxy server between the virtual kubelets and the interlink backend.
IP over Libp2p Functionality
We are testing a new version of the IP over libp2p functionality and developing a proof of concept (POC) for registering publicly resolvable dynamic DNS names for HTTPS access to jobs. This will enable new classes of jobs, such as API endpoints, websites, and server clusters.

These developments in Kubernetes integration are crucial for expanding the capabilities and use cases of the NuNet platform. By connecting Kubernetes with NuNet, we enable more robust and flexible compute solutions, enhancing both platforms’ utility and reach. Stay tuned for more updates as we continue to advance this integration.

NuNet Is Hiring!

NuNet currently has a number of open positions for various roles within the team. If you have the skills and desire to join us in our journey, you can find more information and contact us through our career page.

About NuNet

NuNet lets anyone share and monetize their computing resources, turning cloud computing power from a centralized service into an open protocol powered by blockchain. Find out more via: