A cloud native brew with Oracle Database, Helidon and Kubernetes — Part 2

Published in

Oracle Developers

9 min readMar 18, 2024

In a previous post, I mentioned this new cloud native stack that we are brewing at Oracle that includes the Oracle Database, Kubernetes, and Helidon among others. I don’t have an acronym yet but in the meantime, I want to take you through some of the things we are doing with the Oracle Database and that you might hopefully find useful, especially if you are of a cloud native persuasion.

The ora-operator

Let’s start by taking a look at the Oracle Database Operator. ora-operator is a Kubernetes operator that allows you to handle the lifecycle of your Oracle Database. You can use it to:

manage the lifecycle of all database types on OCI, including Autonomous Database (ADB), Base DBCS running on VMs or baremetal, and Multi-tenant databases (CDBs/PDBs)
manage the lifecycle of Containerized Single Instance Database (SIDB), Containerized Sharded Database (SHARDED) on any Kubernetes cluster where the operator is deployed e.g. OKE
manage the lifecycle of Oracle Data Guard
manage the scaling of the various database flavour
configure the topology and security of provisioned databases on OCI

As long as you have a Kubernetes-compliant cluster, you can run the operator. We’ve tested on both OKE (Oracle’s managed Kubernetes service on Oracle Cloud) and OCNE (Oracle’s on-premise Kubernetes distribution), minikube and in our new-found spirit of openness, we’ve also tested it on Azure Kubernetes Service (AKS), AWS Elastic Kubernetes Service (EKS), RedHat’s OKD and OpenShift.

Let’s get started.

Installing the ora-operator

I assume you already have a Kubernetes cluster of 1 of the above service/distribution. I’ll be using an OKE cluster provisioned through the Terraform OKE Module.

Once provisioned, installing the ora-operator is pretty easy:

Install cert-manager (which you can also install via OKE Addons)
Install the ora-operator

Let’s install the cert-manager:

helm repo add jetstack https://charts.jetstack.io
helm repo update
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.4/cert-manager.crds.yaml
helm install cert-manager --namespace cert-manager --version v1.14.4 jetstack/cert-manager --create-namespace

followed by the ora-operator:

kubectl apply -f https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml

Check the operator is healthy:

kubectl -n oracle-database-operator-system get pods
NAME                                                           READY   STATUS    RESTARTS   AGE
oracle-database-operator-controller-manager-6f4595764f-562zs   1/1     Running   0          49s
oracle-database-operator-controller-manager-6f4595764f-k92dc   1/1     Running   0          49s
oracle-database-operator-controller-manager-6f4595764f-qrjg9   1/1     Running   0          49s

As of now, the ora-operator does not yet support OKE Workload Identity so until this happy day arrives, you’ll have to either configure instance-principal or use a key-based authentication. If you use key-based authentication, I further recommend you set up stronger level of security by using OCI Vault and the CSI Secret Store Driver for OCI instead of just relying on the Secret but that’s for another post. For now, we’ll just use the key-based authentication. Login to your OCI Console and generate a key pair. Ensure you download the private key because you’ll need it in the next step. Then, create a ConfigMap using your credentials:

kubectl create --namespace oracle-database-operator-system configmap oci-cred \
--from-literal=tenancy=ocid1.tenancy.oc1..................67iypsmea \
--from-literal=user=ocid1.user.oc1..aaaaaaaaxw3i...............ce6qzdrnmq \
--from-literal=fingerprint=b2:7c:a8:d5:44:f5.....................:9a:55 \
--from-literal=region=us-phoenix-1

Here, the region refers to where you want to create the database instance.

Using the private key, create a Kubernetes secret:

kubectl create secret --namespace oracle-database-operator-system generic oci-privatekey --from-file=privatekey=~/.oci/oci_api_key.pem

The operator will use this key to authenticate itself with OCI. We are now ready to use the operator to provision databases.

Provisioning an Autonomous Database

For the purpose of this article, I’ll focus on working with ADB but as I mentioned above, you can manage all the flavours of the Oracle Database available on OCI and Kubernetes.

First, let’s create another Secret to hold the password of the admin user. Note that the Secret name and the key must be the same:

kubectl create secret generic admin-password --from-literal=admin-password='password_here'

By default, the Autonomous Database created is set to public. We’ll change that to private. Create a manifest and specify the following and provide the appropriate values for the parameters below:

apiVersion: database.oracle.com/v1alpha1
kind: AutonomousDatabase
metadata:
  name: clouddb
spec:
  details:
    compartmentOCID: ocid1.compartment.....
    dbName: clouddb
    displayName: clouddb
    cpuCoreCount: 1
    adminPassword:
      k8sSecret:
        name: admin-password
    dataStorageSizeInTBs: 1
    networkAccess:
      accessType: PRIVATE
      privateEndpoint:
        subnetOCID: ocid1.subnet...
        nsgOCIDs:
        - ocid1.networksecuritygroup...
      isMTLSConnectionRequired: true
  ociConfig:
    configMapName: oci-cred
    secretName: oci-privatekey

You can find more details on the specs of the ADB on GitHub. Apply the manifest:

kubectl apply -f clouddb.yaml

and if you check the OCI console, you should see the ADB being created:

What just happened here? This diagram below illustrates what happened:

Creating a database on Oracle Cloud with the operator

When you applied the manifest, the ora-operator makes a service call to Oracle Cloud on your behalf. Instead of using the OCI Console, you are now using the manifest to deploy your databases. Since the manifest is basically code, you can use GitOps to manage the lifecycle of the database as we’ll discuss shortly.

Binding an existing Autonomous database

Now that you have provisioned an ADB, you can manage the ADB e.g. you want to scale the number of CPU cores allocated to ADB, enable autoscaling, download wallets and so on. To be able to do that, you must bind your ADB to the Kubernetes cluster. This ADB can be the one you created above or another that was provisioned separately e.g. using OCI Console or Terraform. The neat thing is that you can also bind several ADBs to the same cluster so if you have to migrate your existing applications or build complex applications that span several databases, this can be particularly handy.

Let’s see how this binding works. Create a new manifest ‘adb_bind.yaml’ and enter the following including the ADB’s OCID:

apiVersion: database.oracle.com/v1alpha1
kind: AutonomousDatabase
metadata:
  name: clouddb-bind
spec:
  details:
    autonomousDatabaseOCID: ocid1.autonomousdatabase.oc1.....
    wallet:
      name: clouddb-wallet
      password:
        k8sSecret:
          name: clouddb-wallet-password
  ociConfig:
    configMapName: oci-cred
    secretName: oci-privatekey

You’ll notice a new sub-section in the specs. The clouddb-wallet-password Secret is something you have to create:

kubectl create secret generic clouddb-wallet-password --from-literal=clouddb-wallet-password='<replace_me>'

You can then apply the manifest to perform the binding:

kubectl apply -f adb_bind.yaml

In the above manifest, wallet.password.k8sSecret.name is the name of the Secret (clouddb-wallet-password) that you just created in the step prior and it contains the password required to open the wallet. On the other hand, wallet.name is the name of the Secret (cloudbdb-wallet) which will store the actual wallet. So you should see a new Secret created:

kubectl get secrets
NAME                      TYPE     DATA   AGE
admin-password            Opaque   1      20h
clouddb-wallet           Opaque   9      30s
clouddb-wallet-password   Opaque   1      3m2s

Using the binding feature allows to utilize the full operational power of t̶h̶i̶s̶ ̶b̶a̶t̶t̶l̶e̶ ̶s̶t̶a̶t̶i̶o̶n̶ Autonomous DB to your architectural and operational advantages:

You can manage your core infrastructure, your Kubernetes cluster and your database separately.
You can use and bind several ADBs to the same cluster. This means you can reuse your existing data where they are without copying and duplicating data and then having to maintain a different set of infrastructure and software to keep the data in sync. This reduces a lot of operational overhead, architectural complexity while making your architecture more robust. Oh, it also helps you to keep your costs down.
If you hit a problem with your Kubernetes cluster, you can recreate it using Terraform, Cluster API or using similar Infrastructure As Code technologies and bind the existing ADB to the new cluster. If you are not storing any data, you can even handle Kubernetes like an immutable cluster.
You can isolate your database from your application e.g. you can place your database in a dedicated data infrastructure with the right level of security.

This greatly reduces your burden of having to manage the database as well as the infrastructure and application infrastructure (Kubernetes), and the application. Now, you can mostly focus on the application itself.

Let’s look at some practical examples at how you can use this.

Using the ora-operator with a WebLogic application

Let’s say you are in the process of migrating WebLogic to OCI. You can take advantage of the both ora-operator and the WebLogic Operator to manage the lifecycles of your database and WebLogic servers respectively . Provision your Autonomous Database separately and migrate your data to Autonomous DB using one of the database migration method. You can now take your time and migrate your WebLogic domain to run on OKE, run them concurrently while performing functional and performance tests on OCI, and when you are ready, make the switch. My colleague Omid Izadkhasti has written a few times about using the WebLogic Operator not just to migrate but also to modernize your WebLogic application.

Without the operator, you would have to download the wallet and manually create the Secret to store the Wallet. Now, the operator handles it for you.

Migrate a WebLogic application and its database to OCI

Using the Database in a Helidon application

Now that we’ve got the wallet stored in a Secret, we can also use it in our application running on Kubernetes on OCI. Say you’re now writing a new application in a micro-service style with Helidon, Spring Boot or Node.js and you want to reuse the existing Autonomous database:

Create a binding to the existing Autonomous database in your Kubernetes cluster which will create the Secrets above and all you now have to do is mount the Secret holding your wallet on your pod. Let’s take an example with Helidon.

First, create a Secret from which we can extract sensitive environment variables:

kubectl create secret generic cloudbank-atp \
--from-literal=user=cloudbank \ #schema username
--from-literal=password=<replaceme> \ #password
--from-literal=className=oracle.jdbc.pool.OracleDataSource

Then, create a deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: customer
spec:
  selector:
    matchLabels:
      app: customer
  replicas: 1
  template:
    metadata:
      labels:
        app: customer
        version: v1
    spec:
      containers:
      - name: customer
        image: /uri/to/image
        imagePullPolicy: Always
        env:
        - name: javax.sql.DataSource.cloudbank.dataSource.user
          valueFrom:
            secretKeyRef:
              name: cloudbank-atp
              key: user  #cloudbank
        - name: javax.sql.DataSource.cloudbank.dataSource.password
          valueFrom:
            secretKeyRef:
              name: cloudbank-atp
              key: password #<replaceme>
        - name: javax.sql.DataSource.cloudbank.dataSourceClassName
          valueFrom:
            secretKeyRef:
              name: cloudbank-atp
              key: className
        - name: javax.sql.DataSource.cloudbank.dataSource.url
          value: "jdbc:oracle:thin:@cloudbank_tp?TNS_ADMIN=/opt/creds"
        volumeMounts:
        - name: creds
          mountPath: /opt/creds
        ports:
        - containerPort: 8080
      restartPolicy: Always
      volumes:
      - name: creds
        secret:
          secretName: clouddb-wallet

And deploy:

kubectl apply -f deploy-bind.yaml

Your micro-service should now be able to access the Autonomous Database.

Summary

In this article, we took an introductory look at the ora-operator, a Kubernetes Operator that helps you manage the lifecycle of your Oracle databases on OCI and Kubernetes. With ora-operator, you can not only create or delete an Oracle Database, but you can also bind it to your Kubernetes cluster so you can use it with your application. Creating, deleting or binding your database to use with your application is just the tip of the proverbial iceberg. There are many more architectures and use cases we can unlock with it such as GitOps, the use of immutable Kubernetes clusters as well as improved high availability and resilience. We’ll explore some of these in the next article.

I would like to finish and by thanking my colleagues Julian Ortiz, Shaun Levey, Sherwood Zern and Laird Nelson for their contribution to this article.