The Fastest Route to Hub API: Using the Hub Client in Application Context

The Scramjet Transform Hub (STH) empowers developers to deploy and manage powerful data processing programs. A key tool in this process is the Scramjet SDK. Scramjet SDK allows to easily control your Sequences and Instances, but also in you are using Scramjet Cloud Platform, also your Space and Hubs. This provides powerful ways to implement code execution orchestration and monitoring.

Many existing serverless solutions provide SDK, but Scramjet goes one step further and provides also Scramjet SDK in your application context! This lets you send requests to your STH and Space API internally, without through the proxies, public API gateway or even API server reducing data transfer speed significantly.

Scramjet SDK available in the application context has two instances: Scramjet Hub Client and Scramjet Space Client.

Scramjet Hub Client lets developers send requests internally to STH API. It can be used to start your Sequences, terminate Instances, collect logs and generally perform all STH operations that are exposed through STH API, but much faster and without the need to configure Scramjet SKD Hub Client. This way to you can all possible API requests but only to the STH where the Instance that uses Scramjet Hub Client is deployed.

Another SDK available in the application context — Space Client is even more powerful. It lets developer to control all Space where the Instance that uses it is deployed. From the convenience of your code, without the need to configure Scramjet SDK, you can not only list all Hubs connected to your Space, manage and monitor resources, but also control every Hub in your Space! So you can start Sequences, terminate Instances, and so on, on any Hub that is connected to your Space. You can for example from the cloud supervisor Instance, deploy and start your program on a Hub located on a different cloud, or on the edge environment.

In this blog post we will cover Hub Client, since its mechanisms or working are similar to Space Client but simpler. It will give us a good ground for learning about the powerful Space Client in the next blog post.

A Closer Look at the Implementation

In our example, a JavaScript Sequence is deployed on STH to initiate another Sequence within the same STH. The asynchronous function, which is the main function of this Sequence, aims to start another Sequence identified by `sequenceId`, a parameter passed to it. First the Sequence client object is retrieved using the provided `sequenceId` by utilizing the method `this.hub.getSequenceClient`. Then, the Sequence `start` method is triggered, effectively launching its execution. If successful, the function uses the `this.logger` method to log a confirmation message. In case of errors, it logs the error message and provides detailed error information through instance log endpoint.

const { PassThrough } = require("stream");

module.exports = async function(_input, sequenceId) {
    this.logger.info("Working...");
    const out = new PassThrough();

    try {
        const seqClient = await this.hub.getSequenceClient(sequenceId);

        this.logger.info("Sequence client called:", seqClient);
        const seqStart = await seqClient.start(sequenceId);
        const seqStartStr = JSON.stringify(seqStart);

        this.logger.info(`Sequence ${sequenceId} started: ${seqStartStr}`);
        
        const instClient = await this.hub.instClient(retInstObj._id);
        const instHealth = await instClient.health();
        const instinfo = await instClient.getInfo(); 
        
        this.logger.info(`Instance health: ${instHealth}, Instance info: ${instinfo}`);
        
        out.write(seqStart._id); // write instance id to output stream
        out.end();
    } catch (e) {
        // if error occurs it will be logged under /instance/:id/log endpoint (CLI: si inst log <instID>)
        this.logger.error(e);
    }
    return out;
};

Explanation:

1. Asynchronous Function: The function takes `input` and `sequenceId` as input and initiates the sequence based on the provided Sequence-id.

2. Retrieving Sequence Client: The `hub.getSequenceClient` method fetches the client object associated with the given Sequence-id.

3. Sequence Initiation: The `seqClient.start` method starts the desired Sequence.

4. Handling Results: Upon successful initiation, the Sequence-id is written to the stream and the stream is closed, indicating completion.

5. Error Handling: Catches and logs any errors encountered during the process.

Conclusion

In conclusion, Scramjet SDK offers the flexibility to create a powerful supervisory Sequence that not only monitor other Sequences but also possess the ability to restart them upon failure. This functionality, combined with the capability to initiate and manage other Sequences, makes such a Sequence ideal for automating long-running tasks like data synchronization and scheduled jobs.

Furthermore, Scramjet’s long-lived Instances ensure the supervisor Sequence remains operational throughout the process, providing a reliable and robust solution for managing complex workflows.

Register at Scramjet Cloud Platform for your free trial.