Unveiling NuNet’s Job Orchestration and Scheduling Mechanics

Jennifer Bourke
NuNet
Published in
5 min readMay 8, 2024
Unveiling NuNet’s Job Orchestration and Scheduling Mechanics

How Job Posting and Execution Works in Our Advanced Computing Platform

In today’s rapidly evolving tech landscape, efficiently matching computing jobs with the right resources is critical for performance and scalability. Our platform leverages advanced algorithms and strategies to streamline this process, ensuring that jobs are executed efficiently and effectively. Here’s a closer look at how we handle job postings and executions.

NuNet Detailed Job Orchestration Sequence

Job Posting: The First Step

NuNet Decentralized Computing Job Orchestration

The journey of a computing task begins with a job posting. This is the initial phase where a job is defined and posted to our system. The job details include the specifications of the computing resources required, such as CPU, memory, and runtime environment. This clear description helps in accurately matching the job with the most suitable computing machine.

Search and Match: Finding the Right Machine

Search and Match: Push based search and match

Once a job is posted, the next crucial step is the ‘Search and Match’ process. Our platform employs two main strategies for this — push and pull — each with its strengths depending on the current system load and the nature of the job.

  • Pull Strategy: Here, available machines periodically check for jobs that match their capabilities and opt in to take them up.
  • Push Strategy: On the other hand, this strategy involves actively pushing the job to machines based on predefined criteria, effectively targeting resources that are best suited to the job requirements.
Push based search and match

The goal of these strategies is to efficiently find a machine that is not only capable of handling the job but also optimally aligned with the job’s resource and operational requirements.

Search and Match: Selecting the most preferred node

Job Request

Job request: In this section, the job request will be run for each peer that the user selected as a target in the previous step.

Once we have identified and selected the most appropriate machine for the job, the next step is to send a job request to the compute provider. This request is the formal initiation of the process, where we propose the job details and request acceptance from the provider. This step is pivotal as it transitions from the theoretical matching of resources to practical engagement, setting the stage for formal agreement.

Contract Closure and Job Allocation

Contract Closure and Job Allocation Orchestration

After identifying a suitable machine, we move to the contract closure phase. This is where the details of the job and its execution are finalized with the chosen compute provider. Once the provider accepts the job, it is considered ‘contracted’, and we proceed to the next stages — invocation and allocation.

Invocation and Allocation: Running the Job

NuNet Invocation and Allocation sequence diagram

The invocation and allocation phase is where the job is actually set to run. The job is deployed to the selected machine, and execution begins according to the agreed parameters. This phase is critical as it involves actively monitoring the job to ensure that it executes within the stipulated requirements and handles any issues that might arise during the runtime.

Job Execution

Job Execution on NuNet Platform

The next step in the process is the job execution itself. During this phase, the job runs as specified, utilizing the computing resources of the machine. Our system continuously monitors the execution to ensure efficiency and to quickly resolve any potential interruptions or anomalies.

Throughout each of these phases, our platform ensures that every job is handled with precision, from the moment it is posted to its successful completion. This streamlined process not only optimizes computing resource usage but also significantly reduces downtime and increases overall system productivity.

Contract Settlement and Verification

NuNet Contract settlement sequence diagram

Once a suitable machine has executed the job as per the contract specifications, the next critical step is the contract settlement and verification process. This phase is essential to ensure that the job has been completed satisfactorily and that all contract conditions have been met.

Job Completion Proof:

The compute provider generates proof of job completion, which is essential for contract settlement. This proof typically includes detailed logs, output data, and performance metrics that demonstrate the job was executed as per the agreed terms.

Verification Process:

  • Request for Verification: Once the job is completed, a request for verification of the job completion is sent to the oracle or a designated verification system maintained by NuNet. This request includes all necessary information such as the job description, proof of job completion, and other relevant details.
  • Verification Response: The oracle assesses the provided proof against the contract terms and responds with a verification result. This result determines whether the job is considered verified.

Settlement Actions:

If the Job is Verified:

  • Funds are withdrawn from the service requester’s account (Account 1) and deposited into the service provider’s account (Account 2) as per the contractual terms. This financial transaction is recorded in the contract database, which maintains a secure record of all contractual interactions and settlements.
  • A confirmation message, “Contract fulfilled, thank you for your business,” is sent to both parties, marking the successful completion of the transaction and job.

If the Job is Not Verified:

  • The scenario progresses to a dispute resolution process (not covered in this diagram), which would handle disagreements or failures to meet the contracted standards.

Conclusion

By integrating sophisticated matching algorithms and utilizing flexible job handling strategies, our platform aims to enhance the efficiency of computing resource management. This approach supports a variety of computing demands, helping ensure that operations are handled as smoothly as possible. While we continuously strive to optimize these processes, the complex nature of computational tasks means that improvement is an ongoing journey. We are committed to evolving our system to better meet the needs of our users and adapt to the ever-changing technological landscape.

About NuNet

NuNet is building an innovative, open-source, decentralized computing platform pioneering the new era of DePIN (Decentralized Physical Infrastructure). Find out more via:

--

--