Backend support on Kogito Tooling

Developers can now augment the capabilities of their editors by connecting backend services through an infrastructure that has been included in Kogito Tooling.

Lots of possibilities of executing code outside the client are enabled/facilitated with this new infrastructure, such as calling a java local backend service, reaching an HTTP endpoint, triggering a command in the terminal, to name a few.

Let’s check out how it works in this post.

Photo by Mirko Blicke on Unsplash

Introduction

Backend package

A new typescript package named backend has just been integrated into the Kogito Tooling repository. Composed of a number of sub-modules, this package provides a mechanism for connecting editors to backend services. This package is divided as follows:

• api: Core infrastructure for supporting backend services. All generic API is defined in this sub-module, such as BackendManagerService, BackendProxy, and core related abstractions.
• channel-api: This sub-module should include all backend related APIs that channels can support.
• http-bridge: Bridge implementations between channels and HTTP services. Required for channels supporting such communication. A simple bridge would just bypass the requests whereas a more elaborated strategy could, for example, enqueue the requests, thus having more control over them.
• i18n: Internationalization for backend related features.
• node: Common backend related implementations that target NodeJS.
• vscode: Common backend related implementations that specifically target the VS Code channel. The VS Code API can be used on this sub-module to extend generic services and their capabilities.

Since it is the first iteration of this effort, evolving the infrastructure is expected and encouraged as new services/requirements are being accommodated.

Kogito tooling java repository

A new repository has been created aiming at including java related code with Kogito Tooling. The main idea for having this repository is gathering all the backend services written in java.

The first project pushed into this new repository is a Quarkus app named kogito-extended-services-quarkus, which is part of the infrastructure and basically works as a router for java backend services compatible with Quarkus. In other words, it is only a matter of exposing a REST endpoint that calls the java service to make it available to be accessed by the channels.

Finally, Github actions have been configured to automatically build the code and publish snapshot versions into the Maven central repository on a daily basis. Pull requests also trigger a build workflow.

Typescript infrastructure for channels

The core interfaces of the architecture are Service and Capability. All services are controlled by a backend manager and all capabilities are resolved (and possibly accessed if available) by a backend proxy. There are also some convenient implementations for HTTP services. Let’s check each one of these components in the sections that follow.

Service

A service has its own life cycle, which is controlled by the infrastructure. In other words, the infrastructure starts and stops the services, usually when the service is required the first time and when a channel is stopped, respectively.

Developers can define what should be done when the service starts, stops, and how to check if their requirements are satisfied — if the requirements are not met, the service cannot be started up.

In addition, a service can optionally expose operations to channels through a Capability. The channels, by themselves, can either directly access the capabilities or bridge the communication between editors and services through the envelope.

Capability

A capability is simply an API associated with a service that is implemented and used by channels. The implementation can be either common among channels or distinct ones, depending mostly on the environment.

For example, a capability for doing a particular data processing task could be implemented by using the user’s machine in the case of Desktop and VS Code channels, whereas it could be implemented by accessing some HTTP endpoint in the case of the Online channel. In the end, the implementation is irrelevant for the infrastructure as long as the API is followed by the channels.

Moreover, all API methods provided by a capability should wrap their return in a CapabilityResponse object, which includes a CapabilityResponseStatus. This way, callers of the capability can handle the response in a better way in certain situations, such as the infrastructure or the service itself not being available.

Backend manager service

The backend manager service controls the life cycle of all services. It is supposed to be started up along with the channel and be stopped right before the channel is closed. In the case of the VS Code channel, for example, the backend manager is started up once the extension is activated and stopped when the extension is deactivated (usually right before the VS Code is closed).

The backend manager also provides a mechanism to register both bootstrap and lazy services. If a service needs to be available as soon as the manager is up, then such a service should be registered as a bootstrap service. Conversely, if a service could be started up on the first usage, then such a service should be registered as a lazy service. Finally, all registered services are stopped as soon as the backend manager is stopped.

Backend proxy

The backend proxy simply bridges the access between channels and backend services. Since backend services are optional and/or could be unavailable if their requirements are not satisfied, the backend proxy takes care of resolving the required service, thus letting channels focusing only on the logic of the capability instead of having to deal with service resolving code.

Built-in HTTP services

Services implementing capabilities that require accessing HTTP endpoints can extend the built-in abstract services HttpService or LocalHttpService, which are located in the api sub-module. As the names suggest, the former is meant for any endpoint while the latter is meant for local endpoints that live on the Quarkus app.

Both provide the execute method, which can be used to execute HTTP requests. The only difference between them is the given endpoint:

• Services extending HttpService need to provide a full URL, e.g. http://myserver.com/myservice;
• Since the port of the local Quarkus app is resolved by the infrastructure, services extending LocalHttpService need to provide only the relative endpoint, e.g. /myservice. Behind the scenes, the infrastructure will execute a request on http://localhost:<port>/myservice.

VS Code extension for backend services

Considering that VS Code is the target channel for the first iteration of the backend services infrastructure, a new VS Code extension named vscode-extension-backend has been added into Kogito Tooling. It already includes a basic set up to support the backend manager and Quarkus app, and should register all services associated with VS Code.

This way, backend services on VS Code become optional for users, i.e., the capabilities of the editors can be augmented only if users choose to install the backend extension.

Examples

Now, let’s take a look at some examples running on the VS Code channel to illustrate services and capabilities.

Quarkus local app: Being part of the infrastructure, this service has the sole purpose of starting up the Quarkus app through a java jar command. It does not offer any capabilities for channels. Instead, HTTP endpoints can be exposed inside the Quarkus app to be accessed by channels through other capabilities.

Thus, the life cycle implementations of this service would contain:

• start: run the java jar command to start up the Quarkus app process.
• stop: kill the active process associated with the Quarkus app.
• satisfyRequirements: Possible requirements are (i) check if the jar file of the Quarkus app exists; (ii) check if java 11+ is available; (iii) find an available port. If these conditions are met, then the service can startup.

Quarkus app service on the VS Code channel. The server is started up and stopped by the infrastructure as soon as the extension is activated or deactivated, respectively. Keep in mind that the extension is activated when the first diagram is opened and deactivated only when the VS Code is closed.

Test scenario runner: This sample service has the sole purpose of running a mvn clean test command on a project that contains test scenario files (.scesim), and reporting back the obtained results. Moreover, triggering the command should be done by channels.

Thus, the life cycle implementations of this service would contain:

• start: nothing is necessary to be done on the start phase since the command will be triggered by channels.
• stop: kill the active process associated with the test execution.
• satisfyRequirements: Possible requirements are (i) check if java 11+ is available; (ii) check if maven 3.6.2+ is available. If these conditions are met, then the service can startup.

Since the execution of this service should be triggered by channels, a capability must be associated with the service, which defines a basic API to both trigger and cancel the execution of the test scenarios.

Test scenario runner service on the VS Code channel. Notice that the capability is triggered by the channel through the “Run” button, i.e., the editor does not play a part in this flow.

Quarkus dev runner: This sample service has the sole purpose of running a mvn clean compile quarkus:dev command on a Quarkus project. Similar to the Test scenario runner, triggering the command should be done by channels.

Thus, the life cycle implementations of this service would contain:

• start: nothing is necessary to be done on the start phase since the command will be triggered by channels.
• stop: kill the active process associated with the Quarkus dev execution.
• satisfyRequirements: Possible requirements are (i) check if the given project is compatible with Quarkus; (ii) check if java 11+ is available; (iii) check if maven 3.6.2+ is available. If these conditions are met, then the service can startup.

Since the execution of this service should be triggered by channels, a capability must be associated with the service, which defines a basic API to both trigger and cancel the execution of the Quarkus dev.

Quarkus dev runner service on the VS Code channel. Notice that the capability is triggered by the channel through the “Start Engine” button, i.e., the editor does not play a part in this flow. In addition, a page in the localhost could be opened by the channel side-by-side with the diagram as soon as the engine is available.

Keep in mind that the examples above use NodeJS code to execute the operations on the terminal. As mentioned, services can also reach HTTP endpoints, enabling code in other languages to be run, such as java.

Round trip service sample

Now that we’ve learned about the concepts of the backend services infrastructure, let’s inspect another sample that is meant to establish a communication between the editor and a simple java service. With this sample, we will see how a round trip would be done, starting with the editor and going all the way to a java service.

The java service that will play a part in this sample will simply return a greeting for a given name after a given delay, like Hello, Guilherme!. The intention with the delay is to show that services can take some time to process requests and that’s fine because all the communication is based on promises. Notice that, wherever the flow starts, the associated promise must be handled at some point, either on the channel or in the editor.

The target channel on this sample will be the VS Code, but the same capability implementation could be shared with the Desktop channel. In practice, both channels would be using the local Quarkus app for reaching the java service. On the other hand, Chrome and Online channels would require a slightly different approach to accomplish the same goal, such as having the Quarkus app deployed on some remote server.

Workflow

The workflow of this example will be done as follows:

1. Starting with some action of the user, the editor calls an API available on the envelope with args name="Guilherme", delay=5000. This means that the java service will wait 5000 ms (5 seconds) to return the response;
2. The envelope calls the channel following the associated capability API;
3. The channel accesses the capability through the backend proxy, which resolves the required service with the backend manager;
4. The java service is then accessed through its REST endpoint in the Quarkus app;
5. The java service waits for 5s and then return the response containing the string Hello, Guilherme!;
6. The resolved promise is passed back all the way to the editor;
7. The editor handles the response.

The following diagram summarizes the steps above.

The workflow of the round trip service.

Note: If we draw the workflow of the example services that we’ve seen earlier on this post, like the Test Scenario Runner, the User would be directly connected with the channel since the editor and the envelope do not participate in the flow. Also, the backend manager would access a pure NodeJS service, instead of reaching the Quarkus app.

Implementation

Now, let’s see how we would implement this sample service — code snippets will be provided for the most relevant parts.

Step 1: Capability API

The first step is to define the capability API so all the code can follow. As we’ve seen on the diagram above, the API for this sample service will contain only one method: hello(name, delay).

Step 2: Quarkus service (Java)

The second step is to create the java code that will do all the hard work. Remember that the java code must live on the kogito-tooling-java repository, which is picked up by the building process of the backend VS Code extension.

To get started, the SampleService.java (omitted) is a simple interface that defines our hello method, and, as discussed, the actual implementation should wait for the given delay and then return the response. Here’s how it could be implemented:

Sample service implementation.

In order for channels to be able to access this service, we need to provide a REST endpoint for that. On the following code, HelloRequest.java and HelloResponse.java (omitted) are just DTOs. Here’s how this endpoint could be implemented:

Sample resource accessible through “/sample”.

Now the service is ready to be accessed from its REST endpoint /sample.

Step 3: Editor (Java)

As a third step, let’s focus now on the other edge of this communication: the editor. Since this sample is meant for the KIE GWT editors, the code for this part would live on appformer and kie-wb-common repositories.

The most important code to create on the editor is the API for accessing the envelope. Here’s how this API could be implemented:

Sample capability on the editor.

Notice that, the method signature is pretty similar to the capability API that we’ve defined on Step 1. The only difference is that the return promise object (String) must be wrapped in a CapabilityResponse, which is already available on appformer repository. This way, editors could call and handle the response like in the following code snippet:

Calling and handling service responses.

In addition to that, all classes associated with accessing the envelope must be created, such as NoOpSampleService.java, SampleService.java, SampleServiceWrapper.java, and SampleServiceProducer.java. To accomplish that, there are plenty of examples that can be found here.

Step 4: Envelope and channel code (Typescript)

The fourth and last step is to prepare the communication between the java service and the editor through the envelope/channel. For that, we will need a couple more lines of code, although they are pretty straightforward.

Let’s divide this part into 8 simple inner steps.

Note: To make it easier to understand, the images that follow show the diff between files, and their path is located on the top of the image.

Step 4.1: Set up an identifier for the service. In this case, it will be SAMPLE.

Setting up an identifier for the service.

Step 4.2: Create the typescript capability API compatible with the one that we’ve created in Step 3. Notice that the response (string) is wrapped in a CapabilityResponse object as well.

Creating the capability API.

Step 4.3: Include the method that will bridge the communication between editor and envelope on the CapabilityChannelApi.

Creating the method that will bridge the communication between editor and envelope.

Step 4.4: Create the implementation of the service associated with the capability for the VS Code channel. Notice that this service extends LocalHttpService and implements the capability that we’ve created. Moreover, all life cycle method implementations (start, stop, and satisfyRequirements) are inherited from the LocalHttpService, which can be overridden if necessary.

Creating the implementation of the service for the VS Code channel.

Step 4.5: Expose the capability API on the envelope. The editors can then access it from window.envelope.sampleService through the code that we’ve developed in Step 3.

Exposing the capability API on the envelope.

Step 4.6: Implement the glue code for the editor and the envelope.

Glue code for the editor and the envelope.

Step 4.7: Implement the access to the capability through the backend proxy. In this case, the implementation is being done for the VS Code channel. Since this call will be handled by the editor, the channel only intermediates the call.

Accessing the service associated with the capability through the backend proxy.

Step 4.8: Register the newly created service as a lazy service on the backend manager.

Registering the service as a new lazy service.

After all these steps, the service should be good to go. It means that, once the user interacts with the editor, a call will be triggered and transferred all the way to the java service, which will process and return the appropriate response back to the editor in a promise-based mechanism.

Final remarks

In this post, we’ve seen how the backend services infrastructure for Kogito Tooling is designed. Also, we’ve covered some examples and practical type-safe implementation of a java service that communicates with the editor through the envelope.

Stay tuned because all the possibilities that this new infrastructure enables are yet to come!

And that’s all for today. Thanks for reading! 😃