How to Connect Temporal.io to Astra DB with Kubernetes

This is the third post in a three-part series on how to connect Astra DB — a managed cloud-native database — with open-source Temporal.io to easily manage even your heaviest workloads with the high performance and reliability of Apache Cassandra®.

Temporal.io is an open-source microservice orchestration platform that provides you with a plug-and-play persistence layer that lets you choose and configure your Temporal Server with your preferred backend.

In Part 1 of this blog series on using Temporal.io with DataStax Astra DB, we showed you how Apache Cassandra® is the ideal choice as a reliable and highly available backend database for apps that need scale. In Part 2, we walked you through connecting Temporal with Astra DB using CQL-proxy — a sidecar designed to forward your app’s CQL traffic to an appropriate database service.

In this third and final post of the series, we’ll show you how to deploy Temporal to Astra DB with Kubernetes (K8s). Deploying a K8s cluster is an easy way to test the system and develop Temporal applications. To make things even simpler, we’ll be deploying Temporal to the K8s cluster using Helm Charts, which is great for quick deployment, accessible application configuration, and making sure all your components are running properly — like the app and your Astra DB backend.

That said, let’s get started!

Step 1. Astra DB prerequisites

Before connecting to Temporal, you need to first create an Astra database instance and gather all your credentials. If you haven’t already, register for a free Astra DB account that will give you a generous 80 GB each month to play with. Next, follow these instructions to spin up your database:

Create an Astra database

  • When asked for the keyspace name, name it temporal.
  • Once your database is created, navigate back to your database dashboard and click Add Keyspace.
  • Name this second keyspace temporal_visibility .
  • The status of your database will temporarily go to Maintenance, but after a couple seconds you can refresh your screen and the status should go back to Active.
Figure 1. Screenshot of Astra DB dashboard and where to add the keyspaces.

Create an Astra Token

Tokens are required to authenticate against Astra with APIs or Drivers. These tokens can be used with multiple databases and can be configured to have specific permissions. In this example, you will create a token with Admin Role.

Temporal uses this token to receive credentials and permission to access your database in a similar way to how Cassandra has a “user” and “password”, which we’ll discuss in more detail in Step 4 where you will configure the Persistence Layer in Temporal.

When you create your tokens, download the CSV file to keep these credentials.

Download your secure connect bundle ZIP

Download the secure connect bundle for the database that you created specifically for Temporal. These are unique to each database that you create within your Astra organization. This contains the information for Temporal to initialize a secured TLS connection between it and your Astra DB instance.

Find your Database ID

Lastly, you want to get your Database ID. You can find this in one of two ways:

  1. Navigate to your your database and get the last ID in the URL: https://astra.datastax.com/org/.../database/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
  2. Copy and paste the “Datacenter ID” without the trailing -1 from the “Regions” section of your Astra Dashboard.

We recommend you create a central directory somewhere in your electronic filing system to keep track of all the files needed for this setup. For this example, we’re calling this main directory /my-temporal, the Secure Connect Bundle is secure-connect-temporal, and the CSV for token credentials is saved under adminuser (see Figure 2).

Figure 2. Example of a central directory to keep track of all the files needed for this setup.

Step 2. Temporal schema migration

Temporal offers a few ways to quickly install and deploy their server. For this example, we’re going to use both Docker and Helm Charts to get us connected with Astra DB. Follow the

prerequisites (and just the prerequisites) for both Docker and Helm Charts to get you set up. In this step, you’ll build and set up the keyspaces you created earlier in Astra DB with temporal-cassandra-tool. This tool is part of the Temporal repo and relies on the schema definition.

  • Clone this GitHub repository.
  • Navigate to your cloned repository and using your preferred text editor (e.g. VisualStudio or Sublime), update the .env file with the Astra Token and Astra Database ID that you obtained above.
  • Navigate to your cloned temporal-astra-cql-proxy directory.
  • Run the following commands to initialize the keyspaces that you created through Astra. Note that there are two sets of commands: one for temporal keyspace and one for temporal_visibility keyspace.

Once the process is complete, you should see a message similar to this:

  • Lastly, run the following command to update the schema version being used. This will make sure that Temporal server and Astra DB are using compatible versions.

Great! Your schemas have been migrated with Astra DB. You can double-check to make sure the correct tables have been created by querying your database in Astra DB’s CQL Console.

If you do a quick DESCRIBE tables in both your temporal and temporal_visibility keyspaces, you should see there are tables loaded in that were created by the schema migration.

Step 3. Deployment configuration

In this section, you’re going to install Temporal using Helm Charts. Here’s how:

  • Refer to the Prerequisites that gives you the steps in configuring your system to access a K8s cluster, AWS CLI V2, kubectl, Helm v3, and so on. For this specific example, we used Kind to deploy our K8s cluster.
  • Clone the temporalio/helm-charts repository in your /my-temporal directory.
  • Download Helm Chart Dependencies. Your output should look like this:
  • Create and deploy your K8s cluster using these instructions. Once deployed, the K8s cluster should appear up and running in your Docker Desktop.
Figure 3. Verify in the Docker Desktop that your Kubernetes cluster is up and running.

Secret creation

In the Docker Image step, you might have noticed that we used the following files from our Astra secure connect bundle: ca.crt, cert, and key. These are all credentials that will tell Temporal that you’re trying to connect with Astra DB and that you have the access to do so.

With Secrets in K8s, you can store sensitive information and mount it to your deployment.

  • Go to your /helm-charts file directory.
  • Open the templates folder.
  • Using your preferred text editor (we used VSCode) create an astra-secret.yaml file in this folder.
  • Copy and paste the following code in your editor:
  • Navigate to your Secure Connect Bundle in your /my-temporal directory using your terminal.
  • Base64 Encode your ca.crt, cert, and key files then plug them back into the respective fields in your astra-secret.yaml file ex. ca.crt: LS0tLS1CRUdJTiBDRV
    – Base64 for Mac: base64 — break 0 <file-name>
    – Base64 for Windows: certutil -encode input-file.txt encoded-output.txt
    – Base64 for Linux: base64 -w o <path-to-file>
    – Save your astra-secret.yaml file.

Mounting to volume — server deployment

  • In your templates folder, find and open server-deployment.yaml
  • Append the following under volumeMounts:
  • Scroll down to volumes and append the following:

Note that the volumeMount name and volume name must be the same for astra-secret.

  • Lastly, save your server-development.yaml file.

Step 4. Persistence layer configuration

Now that you’ve gotten this far, we’re going to configure the Persistence Layer of Temporal’s server. The persistence layer, also known as the Data Access layer, is paired with the Temporal Server to allow support for different back-end databases — Cassandra, in this case.

  • Go to the helm-charts directory.
  • Go to the values folder.
  • Edit the values.cassandra.yaml file using your preferred editor (i.e. VSCode).
  • Modify the contents under the cassandra sections for both temporal and temporal_visibility keyspaces as shown below.
  • Save your values.cassandra.yaml file.

Helm install

Now it’s time to install and deploy Temporal Server.

  • In your helm-charts directory, run this command:

You should see this message if it was successful:

  • To verify that Temporal has started, in a different tab run kubectl — namespace=default get pods -l “app.kubernetes.io/instance=temporalastra”

A successful deployment should look like this:

  • Allow a couple minutes for all STATUS to initialize and switch to Running. You can run the get pods statement from above to continuously check on the status as it’s initializing.

Step 5. Test and validate

You can test your connection and play with your Temporal cluster by following these instructions on GitHub.

  • Make sure to use tctl to create namespaces dedicated to certain workflows.
  • Forward your machine’s local port to Temporal Web UI to view and access on your local host by running the following commands in a new window:
  • When using the sample apps, keep in mind that you want to modify the starter and worker code so that it points to this specific Temporal deployment. For example:
  • Once you have this all running, you should be able to see your workflows reflect on both the Astra UI and the Temporal UI by navigating to http://127.0.0.1:8088 on your browser.
Figure 4: Verifying that your workflows show up on both the Temporal UI and Astra UI.

And you’re all done!

At this point you should have a better idea of just how powerful a pairing Temporal and Astra DB can be. With this runbook and your free Astra DB account, you can go ahead and start experimenting with Temporal samples. If you have any interesting app ideas in mind using Temporal and Astra DB — tweet us @DataStax Developers!

Subscribe to the DataStax Tech Blog for more developer stories like this. For free tutorials and workshops, check out our DataStaxDevs YouTube channel.

Resources

  1. Part 1: Easily Manage Workflows at Scale with Temporal.io and Astra DB
  2. Part 2: Connect Temporal.io to Astra DB in 5 Easy Steps with CQL-Proxy
  3. DataStax Astra DB | Cloud-Native Cassandra-as-a-Service
  4. How to quickly install a Temporal Cluster for testing and local development
  5. Temporal Helm charts
  6. Kubernetes

--

--

--

We’re huge believers in modern, cloud native technologies like Kubernetes; we are making Cassandra ready for millions of developers through simple APIs; and we are committed to delivering the industry’s first and only open, multi-cloud serverless database: DataStax Astra DB.

Recommended from Medium

How to Get Cloud Migration Right

If Google Has Started Today…

The Toit language is now open source

Message Queues in AWS SQS

Journey of Learning Solidity — Tips And Tricks From Java Developer’s Perspective

Automating the clinical coding process in an NHS Trust

Live Media Streaming

I don’t like object oriented programming anymore

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
DataStax

DataStax

DataStax is the company behind the massively scalable, highly available, cloud-native NoSQL data platform built on Apache Cassandra®.

More from Medium

The Next Generation Database

A Beginner’s Guide to Benchmarking with NoSQLBench

Kafka on Kubernetes: Using Strimzi — Part 4 (Scalability)

Tekton — Kubernetes Native CI/CD