Running and scaling the "Hello World" integration flow using the Terminal is soooo cool
Don't be afraid. The command line has many secrets.
In this post I will present you to the command line utility, used to interact with a Digibee realm. Overall, this tool provides users with a powerful and flexible interface to interact with Digibee's low-code and cloud native integration platform, enabling automation, scripting, remote access, and integration with other tools. It enhances efficiency, portability, and reproducibility while catering to diverse user needs and scenarios.
Some Context
In a bustling tech startup, there was a talented and resourceful developer named Alex. Their company relied heavily on an Integration Platform as a Service (iPaaS) solution to connect and streamline data flows between various systems. However, as the company grew and the data volume increased, the integration microservice started experiencing performance bottlenecks, posing a significant challenge.
Recognizing the urgency of the situation, Alex turned to their trusty command line utility, armed with deep knowledge of the iPaaS system and an innate problem-solving ability. They embarked on a mission to analyze the performance patterns and identify the root cause of the bottlenecks.
Alex decided to retrieved crucial metrics such as CPU and memory usage, network throughput, and response times using a monitoring interface. This data analysis revealed that the integration microservice was hitting resource limits, resulting in delays and degraded performance. It became evident that scaling the microservice was crucial to meet the growing demand.
Drawing upon the automation capabilities of the command line utility, Alex devised a meticulous plan to dynamically scale the integration microservice. They crafted scripts that leveraged the utility’s scaling commands, enabling automatic adjustments to the number of micro service instances based on real-time workload metrics.
Do you recognize this type of situation? Let’s talk about it and discover Digibee’s approach to overcoming these challenges with Digibeectl.
What is Digibeectl?
Digibeectl is an API-enabled command-line tool (CLI) that serves as a game-changing solution for orchestrating deployments in an efficient and user-friendly manner. It provides a range of features and capabilities to facilitate pipeline management and automation. With Digibeectl, users can easily create flexible pipelines and interactively scale their integrations as needed.
Let's Start!
Before playing with Digibeectl, we need to understand some basic concepts:
Digibee Deployment
After finishing the development phase of a pipeline, the next step is to promote the "low-coded" integration to a running environment, TEST or PROD. We do this by creating a deployment. During this step, the developer needs to assign computational capabilities that will be used by the integration flow microservice, to process messages. These capabilities are represented by three available sizes.
A deployment is basically a pipeline compiled and executed as a microservice, that runs on the Digibee Integration Platform.
When creating a deployment, the developer can choose the scale of a pipeline by giving it vertical and horizontal resources. Bellow are the variables that can be adjusted during the build phase:
- CPU and Memory — selecting the size SMALL, MEDIUM or LARGE
- Concurrent Executions — selecting the number of executions and limited by size
- Replicas — selecting how many replicas of the previous two configurations will be running
How to install Digibeectl?
Digibeectl is provided as a ‘tar.gz’ file for MacOS and linux. The command below helps in the digibeectl installation with one execution only. For that, open a terminal window and execute:
curl -s https://storage.googleapis.com/digibee-release-test/releases/install.sh | bashSupported Platforms
In general, digibectl supports all available modern linux distros running over arm64 or x86 (arm is not supported). If you want to check all the available binaries, follow this link:
According Digibee official documentation, digibeectl doesn’t support Windows binaries at this moment. But you can run it on Linux over Windows by using WSL (https://learn.microsoft.com/pt-br/windows/wsl/install) or any Linux running over a virtualization technology, like Virtualbox or VMware.
Configuring Digibeectl
Digibeectl requires a configuration file that is encrypted and an authentication token. To generate this file you need to open Digibee’s Interface.
The first thing to do is to navigate the interface click in the realm governance related configurations and find, in the left-side, the Digibeectl option in the menu.
After that you need to give it a name, select the permissions and the expiration time.
Follow the instructions described in the:
digibeectl set config --helpUse the following command line to configure digibeectl:
digibeectl set config --file ./path/my.config.json --secret-key "xxxxxxxx" --auth-key "yyyyyyyyy"The digibeectl save its configuration on the following path:
$HOME/.digibeectl/config.jsonSwitching Between Realms
In its actual version, digibeectl doesn’t support a multi-realm environment on the same Linux or Mac user account. Different users can have different tokens, each one configured to communicate with one specific realm. As the configuration is stored on the $HOME/.digibeectl/config.json you can switch between realms using some approaches:
- Configure digibeectl with a token -> Rename the config.json to config-token1.json;
- Configure the digibeectl with another token -> Rename the config.json to config-token2.json;
- If you wish to use the token1, copy config-token1.json to config.json;
- If you wish to use the token2, overwrite the config. json with the config-token2.json;
- Use the dgb-realmswitch
Configuring and Using the dgb-realmswitch
To use dgb-realmswitch you need to:
- Access the repo: https://github.com/arturscheiner/dgb-realmswitch
- Read the easy-to-follow documentation available on the project page
After following the instructions the dgb-rs utility will be installed on your computer. With the command bellow you can access its help information.
dgb-rs -h
What you can do with Digibeectl?
This small piece of software can help a lot, but it have its limitations. In its actual version, this CLI tool has its main focus on the RUN. Yes, on the RUN… aka DEPLOYMENT. Gotcha?
Like in build, RUN and monitor …
Although you have some limited pipeline BUILD related commands, it is at the RUN department that it shines! With it, you will be able to create, scale up, scale down and delete deployments.
Furthermore with Digibeectl you will have the power to redeploy deployments without disrupting the integrations. Just resize them on-the-fly … like you do using the RUN GUI, but from the CLI.
With digibeectl configured on your computer, write on your terminal screen the following command:
digibeectl create deployments -h
And you will get the help information, like in the image above.
Deploying an Integration Pipeline
Now that we have everything set, it is time to deploy a pipeline. In this example we will deploy the hello-world-pipeline that we have created on the previous post. You can find the post clicking in the link bellow:
Before deploying the hello-world-pipeline let's first check if is deployed on the configured realm:
digibeectl get deployment --name hello-world-pipeline
As you can see, the pipeline is not deployed. So, to accomplish the proposed task we must use digibeectl to get the pipeline id. The pipeline id is needed by the deployment command.
digibeectl get pipelines --name hello-world-pipeline
With the pipeline id, we can deploy the pipeline using the command bellow:
digibeectl create deployment --pipeline-id af5e049b-dfc3-4dcc-8909-97b7a8ef2877 -e test --replicas 1 --pipeline-size SMALLAs you can see the command specifies that we will create a micro service of the size SMALL (default 10 consumers/threads) with one replica. The return of this command is a simple "Deployment created" message. Now let's check if it is running:
digibeectl get deployment --name hello-world-pipeline
Scaling Up the Integration Pipeline
Now that we have our hello-world-pipeline running, it's time to scale it. We will change its size from SMALL to MEDIUM with two replicas. The command needed to accomplish this is describe bellow:
digibeectl create deployment --pipeline-id af5e049b-dfc3-4dcc-8909-97b7a8ef2877 -e test --replicas 2 --pipeline-size MEDIUM --redeployNote that this command has a redeploy parameter specified. The return is the same as before, "Deployment created". Now let's check the status of the deployment with the command:
Impressive! We expanded our integration pipeline on-the-fly and now it has twice the capacity and an extra replica. In addition to providing horizontal resources, a replica also extends the high availability of the individual pipeline. In this case, two microservices, representing the same flow, is running and available to process messages at the same time.
To scale down the pipeline you can use the same logic. Just change the attributes and … be happy! 😁
SUMMARY
In this post I showed you how to deploy the “Hello World” integration pipeline using the terminal command line utility, digibeectl. This utility can be integrated with other tools and with some creativity allows users to automate tasks and build scripts for repetitive or complex operations. It provides a way to interact with the Digibee low-code platform programmatically, enabling the integration of platform operations into existing automation workflows and processes.
So… stay tuned! I will be releasing posts that will shows you:
- How to RUN pipelines from the GUI interface
- And how to MONITOR them
Besides that, I will release other tutorials that explore some nice techniques like:
- How to use DIGIBEECTL together with bash to extract a pipeline flowSpec and sync it with a git repo in order to track changes within a DevOps CI/CD platform
- How to create a custom CAPSULE to be used in a pipeline, the same way you use a connector
If you are a curious person and would like to know more about Digibee, check out the digibee.academy. There you will find trainings and certifications. You can also find plenty of resources on Digibee website.
Thank you for your time and reading, I hope to see you in the next posts!
