Open in app

Sign in

Write

Sign in

Provision and configure Oracle Integration Cloud instance using Terraform

Omid Izadkhasti
Oracle Developers
Published in
11 min readAug 7, 2023

With an increase in DevOps adoption, most companies like to provision all cloud resources (include IaaS and PaaS resources) using automation scripts. Terraform is one of biggest players in infrastructure automation.

Most of Oracle Integration Cloud (OIC) customers would like to automate provisioning and configuration of OIC instances. In this article we’ll use Oracle Cloud Infrastructure Terraform provider features, to provision and configure OIC instances in OCI.

This looks like a simple activity, but you will see there are few challenges we need to address.

You can use this information later to automate provisioning of OIC instances in different regions (for example: provision a DR instance).

Prepare OCI Terraform provider

First you’ll need to provision the OCI terraform provider. Create following terraform files below. One of the files contains all variables that we need to send to the terraform provider (OCI tenancy information) and the second file is an OCI terraform provider configuration.

provider "oci" {
  alias                     = "home"
  region                    = var.home
  tenancy_ocid              = var.tenancy_ocid
  user_ocid                 = var.user_ocid
  fingerprint               = var.fingerprint
  private_key               = var.private_key
}
variable "tenancy_ocid" {
  description = "Tenancy OCID"
  default = "ocid1.tenancy.oc1......."
}

variable "user_ocid" {
  description = "User OCID, this user should have permission to provision resources inside parent compartment"
  default = "ocid1.user.oc1......."
}

variable "fingerprint" {
  description = "Private key fingerprint"
  default = "<OCI Private Key Fingerprint>"
}

variable "private_key" {
  description = "Private key"
  default = <<EOT
-----BEGIN RSA PRIVATE KEY-----
.
.
-----END RSA PRIVATE KEY-----
EOT

}

variable "home" {
  description = "Home region name"
  default = "us-ashburn-1"
}

You need to create an OCI signing key pair and upload the public key to the user that you want to use in terraform (please follow this link for more information).

After updating all variables you can initialize a working directory that contains these configuration files using the following command (I assume you already installed the terraform in your local machine, if not please follow this link).

terrafom init

Now you can create another terraform file to provision the OIC instance.

Provision OIC instance using terraform

There is a terraform resource for Oracle Integration Cloud (oci_integration_integration_instance). Use following sample terraform file to provision an OIC instance.

variable "compartment_ocid" {
  description = "OCID of the Compartment that you want to provision the OIC instace"
  default = "<compartment OCID>"
}

variable "oic_instance_name" {
  description = "OIC instance name"
  default = "oic-instance"
}

variable "oic_license_type" {
  description = "License type NEW/BYOL"
  default = "NEW"
}

variable "oic_file_server" {
  description = "OIC file server enable/disable"
  default = "true"
}

variable "oic_visual_builder" {
  description = "OIC Visual Builder enable/disable"
  default = "true"
}

variable "oic_instance_edition" {
  description = "OIC instance Edition OIC Gen 2 STANDARD/ENTERPRISE
                 OIC Gen3 STANDARDX/ENTERPRISEX"
  default = "ENTERPRISEX"
}

variable "oic_instance_message_packs" {
  description = "OIC instance number of message pack"
  default = "1"
}

variable "update_oic_token" {
  default = "1"
}

variable "build_oic_agent" {
  default = "true"
}

variable "idcs_token" {
  description = "IDCS Authentication token"
  default = "<IDCS token>"  
}

resource "oci_integration_integration_instance" "oic_instance" {
  compartment_id            = var.compartment_ocid
  display_name              = var.oic_instance_name
  integration_instance_type = var.oic_instance_edition
  is_byol                   = var.oic_license_type == "BYOL" ? "true":"false"
  message_packs             = var.oic_instance_message_packs

  is_file_server_enabled = var.oic_file_server
  is_visual_builder_enabled = var.oic_visual_builder
  state                  = "ACTIVE"

  idcs_at = var.idcs_token

}

As you can see, you need to update the OIC instance variables before executing the terraform script (for example: instance name, instance compartment, number of messages pack for the instance, instance license type, instance edition, enable file server and enable visual builder).

Note: You can use integration_instance_type parameter to determine the version of an OIC instance. You can use STANDARD/ENTERPRISE to provision OIC Gen2 and use STANDARDX/ENTERPRISEX to provision OIC Gen3.

There is one interesting variable that you need update before executing the terraform script called ‘idcs_at’.

OIC is a PaaS service and integrated with OCI Identity Domain (which used to be Identity Cloud Service — IDCS). Because you can have several identity domains (or IDCS stripes) in your tenancy, you should tell OCI terraform provider in which identity domain (or IDCS instance) you want to provision your OIC instance. You can manually download identity domain authentication tokens using the following instructions and update terraform variables using the token value.

  • Navigate to OCI console
  • Navigate to Identity & Security -> Domains
  • Select the domain you want to provision for your instance.
  • Select “Applications” from the navigation menu
  • In the Applications page, click “Add Application”
  • Select Confidential Application and click “Launch Workflow”
  • Click “Configure this application as a client now” and select Client Credential and JWT assertion grant types.
  • In the Token issuance policy section, select “Add app roles” and add “Identity Domain Administration” role.
  • Activate the application.
  • Click on “Access Token” under resources and select download token.

Alternately, you can use the Rest API to download this token using Client ID and Client Secret of this confidential application using the following command.

You can find Client ID and Secret from this page
You can find Domain URL from here
curl --location '<Domain URL>/oauth2/v1/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--header 'Authorization: Basic <BAse64 encoded of ClientID:ClientSecret>' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'scope=urn:opc:idm:__myscopes__'

When you download the token, you need to capture access_token value (token format is similar below).

{
    "access_token": "<Access token data>",
    "token_type": "Bearer",
    "expires_in": 3600
}

Note: The token has an expiry date and will expire after some time (expires_in field).

You can simply pass access_token value to idcs_at parameter in OIC terraform resource or use following code to read token file (filename is token in this example) and pass the access_token to this parameter.

resource "oci_integration_integration_instance" "oic_instance" {
  compartment_id            = var.compartment_ocid
  display_name              = var.oic_instance_name
  integration_instance_type = var.oic_instance_edition
  is_byol                   = var.oic_license_type == "BYOL" ? "true":"false"
  message_packs             = var.oic_instance_message_packs

  is_file_server_enabled = var.oic_file_server
  is_visual_builder_enabled = var.oic_visual_builder
  state                  = "ACTIVE"

  idcs_at = jsondecode(data.local_file.idcs_token.content)["access_token"]

}

data "local_file" "idcs_token" {
  filename = "token"
}

Finally, execute the following command to provision the OIC instance.

terraform plan
terrafom apply

This solution works perfectly to provision OIC instances, but if you want to automate this process to provision OIC instances using DevOps tools you still need the manual step of downloading the token and updating the terraform variable file.

In next section I will explain how to automate this process in terraform.

Automate access token generation using terraform

In this section I am going to use some of the terraform capabilities like local-exec provisioner to generate an access token and pass it to integration resource to provision the OIC instance.

Here is the code to use the curl command inside our terraform script to generate a token and use it in an integration resource.

variable "domain_url" {
  default = "<Domain URL>"
}

variable "clientid" {
  default = "<Identity domain confidential application client id>"
}

variable "clientsecret" {
  default = "<Identity domain confidential application client secret>"
}

resource "null_resource" "get_idcs_token" {
  
  provisioner "local-exec" {
      command = <<EOT
        curl -X POST  ${var.domain_url}/oauth2/v1/token -H 'Authorization: Basic ${base64encode(format("%s:%s",clientid,clientsecret))}' -d 'grant_type=client_credentials&scope=urn:opc:idm:__myscopes__' | jq -r ".access_token" > idcs_token
EOT
  }
}

This code accepts domain URL, confidential application (that we created in the previous step), client ID and secret and uses the identity domain REST API to generate the token.

Also, I recommend putting client id and secret (sensitive information) inside an OCI secret and use terraform resource to read these secrets during runtime. For example, you can use the following terraform code to read secrets from OCI vault.

variable "client_id" {
  default = "<Secret OCID of Client ID>"
}

variable "client_secret" {
  default = "<Secret OCID of Client Secret>"
}

data "oci_secrets_secretbundle" "clientid_secretbundle" {
    secret_id = var.client_id
    stage     = "CURRENT"
}

data "oci_secrets_secretbundle" "clientsecret_secretbundle" {
    secret_id = var.client_secret
    stage     = "CURRENT"
}

#Read secret bundle content and store it in local variable and decode from Base64 format
locals {
  clientid = base64decode(join(",",data.oci_secrets_secretbundle.clientid_secretbundle.secret_bundle_content[*].content))
  clientsecret = base64decode(join(",",data.oci_secrets_secretbundle.clientsecret_secretbundle.secret_bundle_content[*].content))
}

Automate provisioning of connectivity agent

Configuration of connectivity agent is another task that we can automate. We need to provision a compute instance (inside private or public subnet depending on your architecture), download connectivity agent installer from the OIC instance, unarchive installer and configure connectivity agent with OIC instance configuration (by updating InstallerProfile.cfg file) and finally execute the agent.

We can automate almost all these tasks. First we need to update our terraform code and include provisioning of network resources (VCN, subnet, gateways, security lists and routings) and VM instance to host the connectivity agent (I will skip this part and you can follow OCI documentation and Terraform OCI provider documentation to do it).

Next, we need to execute a script in our compute instance. Before creating the script, we need to create the identity domain user and add user to a specific role in the OIC instance (for example ServiveAdministrator role).

You can navigate to Domains in OCI console and select “Oracle Cloud Services” from the left hand menu and select the OIC instance that you already provisioned.

Navigate to “Application Roles”, select the role that you want to assign your user (for example “ServiceAdministrator” and add your user.

Also, you can automate this activity using Identity Domain Rest API’s if you like (this is out of scope for this post).

Now, your user has permission to download connectivity agent installer from OIC instance using OIC Rest API’s.

Here is sample Python script that you can use to automate configuration of connectivity agent on a compute instance.

echo 'Shell script to prepare OIC agnet computer'
oicUsername=$1
oicUserPassword=$2
oicInstanceURL=$3
oicAgentGroupID=$4
userName="oracle"
groupName="oracle"
homeFolder=$(echo "/home/"$userName)
oicUrl=$(echo $oicInstanceURL | cut -d"/" -f1-4)

#Create oracle user and group if not exists (Install and execute agent under oracle account) 
if [ ! $(getent group $groupName) 2>/dev/null ]; then
   sudo groupadd "$groupName"
fi
if [ getent passwd $userName > /dev/null 2>&1 ]; then     
    echo "User oracle exists"
else 
    sudo useradd -m -g "$groupName" -d "$homeFolder" -s /bin/bash "$userName"
fi

#Create agent folder
folder="/home/oracle/agent"
if [ ! -d "$folder" ]; then
  sudo mkdir -p $folder
  sudo chown -R $userName:$groupName $folder
fi

#Download cnnectivity agent from OIC instance in /home/oracle/agent folder
downloadFile=0
if [ -d /home/oracle/agent ] ; then
    if [ -f /home/oracle/agent/oic_connectivity_agent.zip ] ; then
        fileSize=10000
        fileName="/home/oracle/agent/oic_connectivity_agent.zip"
        if [ $(ls -l "$fileName" | cut -d " " -f5) -lt $fileSize ] ; then
            downloadFile=1
        fi
    else
        downloadFile=1     
    fi
else
   downloadFile=1
fi        
if [ "$downloadFile" = 1 ] ; then
    oicAgentDownlodUrl=$(echo $oicUrl/api/integration/v1/agents/binaries/connectivity)
    curl -u $oicCredential $oicAgentDownlodUrl > /tmp/oic_connectivity_agent.zip
    chmod 755 /tmp/oic_connectivity_agent.zip
    sudo runuser -l $userName -c 'cp -rf /tmp/oic_connectivity_agent.zip /home/oracle/agent'    
fi

# Create OIC agnet group using OIC Rest API
agentGroupName=$oicAgentGroupID
agentGroupID=$(echo $oicAgentGroupID | tr '[a-z]' '[A-Z]')
agentExists=$(curl -X GET   -u '$oicUsername:$oicUserPassword'  $oicUrl/api/integration/v1/agentgroups/$agentGroupID &>/dev/null | jq .name | sed 's/\"//g' | wc -l)
oicCreateAgentGroupUrl=$(echo $oicUrl/api/integration/v1/agentgroups/$agentGroupID)

if [ $agentExists = 0 ]; then
    curl -X POST -u $oicCredential $oicCreateAgentGroupUrl --form name=$agentGroupName
fi    

#Install JDK
jdk_installed=$(sudo yum list installed 2>/dev/null | grep -c java-11-openjdk)
if [ "$jdk_installed" = 0 ]; then
    sudo yum -y install java-11-openjdk java-11-openjdk-devel
fi
java_home=$(dirname $(dirname $(readlink $(readlink $(which java)))))

#Install agent
if [ ! -d /oracle/agent/agenthome ]; then 
    sudo runuser -l $userName -c 'cd /home/oracle/agent ; unzip oic_connectivity_agent.zip'
    sudo runuser $userName -c 'agentConfigFile="/home/oracle/agent/InstallerProfile.cfg";
    agentConfigFileNew="/home/oracle/agent/InstallerProfile.new";
    oicUrl=$(echo '$oicInstanceURL' | cut -d"/" -f1-3);
    while IFS= read -r line
    do
    if [[ $line == *"oic_URL="* ]]; then
        echo $line$oicUrl >> $agentConfigFileNew
        echo "oic_USER="'$oicUsername' >> $agentConfigFileNew
        echo "oic_PASSWORD="'$oicPassword' >> $agentConfigFileNew
    elif [[ $line == *"agent_GROUP_IDENTIFIER="* ]]; then 
        echo $line"'$agentGroupID'" >> $agentConfigFileNew
    else
        echo $line >> $agentConfigFileNew
    fi

    done < $agentConfigFile
    cp -rf $agentConfigFileNew $agentConfigFile'
fi

#Start Agent
getStatusOfAgent=$(ps -aux | grep connectivityagent.jar | grep -v grep | wc -l)
if [ $getStatusOfAgent = 0 ]; then
    sudo runuser $userName -c 'cd /home/oracle/agent; export JAVA_HOME='$java_home'; export PATH=$PATH:$JAVA_HOME/bin;
    nohup java -jar connectivityagent.jar &'
fi

sleep 5
#Get Agent Status
agentStaus=$(curl -X GET   -u $oicCredential  $oicUrl/api/integration/v1/monitoring/agentgroups/$agentGroupID/agents | jq '.items[].displayStatus' | sed 's/\"//g')
echo "Agent Status is "$agentStaus

This script accepts OIC instance URL, OIC Username, OIC user password and OIC agent group ID as parameter and does the following tasks on compute instance:

  • Creates oracle user and group (we don’t want to install agent under opc user).
  • Creates agent folder (/home/oracle/agent).
  • Downloads agent installer from OIC instance using /api/integration/v1/agents/binaries/connectivity Rest API.
  • Creates agent group in OIC instance using Rest API.
  • Installs JDK.
  • Unzips agent installer, update InstallerProfile.cfg file with instance URL, OIC Username/password and agent group.
  • Executes agent (nohup java -jar connectivityagent.jar &).
  • Checks OIC agent status using Rest API.

Finally, we can use following terraform code to execute the script on our compute instance (directly or over a Bastion host).

resource "null_resource" "remote-exec" {

  //copy script file to oic-agent compute
  provisioner "file" {
 
    source = "${path.root}/scripts/oicAgentConfigurationScript.sh"
    destination = "/tmp/oicAgentConfigurationScript.sh"

    connection {
      agent       = false
      timeout     = "30m"
      host        = var.oic_agent_private_ip
      user        = "opc"
      private_key = var.oic_agent_private_key
      bastion_host = var.bastion_public_ip
      bastion_user = "opc"
      bastion_private_key = var.bastion_private_ip
      } 
    }
    
  //execute script on oic agent compute
  provisioner "remote-exec" {
    connection {
      agent       = false
      timeout     = "30m"
      host        = var.oic_agent_private_ip
      user        = "opc"
      private_key = var.oic_agent_private_key
      bastion_host = var.bastion_public_ip
      bastion_user = "opc"
      bastion_private_key = var.bastion_private_ip     }

    inline = [
      "chmod 755 /tmp/oicAgentConfigurationScript.sh",
      "/tmp/oicAgentConfigurationScript.sh ${var.oic_user} ${var.oic_password} ${var.oic_instance_url} ${var.oic_agent_group_id}",
    ]
  }
}

I am using the terraform file and remote-exec provisioner to transfer the script to the compute instance and execute the script. To connect to a compute instance, you need the private key and private IP of compute and public IP and private key of the Bastion host (if you are using compute with public IP you can directly send and execute the script without using Bastion host).

Conclusion

In this article, I have explained simple steps to use terraform and identity domain REST API’s to automate provisioning of an OIC instance and connectivity agent. You can simply extend this code to automate provisioning of other resources (like API gateway and configure it inside OIC, etc.).

References

OCI Terraform Provider: https://registry.terraform.io/providers/oracle/oci/latest/docs

Terraform Oracle Integration Resource: https://registry.terraform.io/providers/oracle/oci/latest/docs/resources/integration_integration_instance

OCI API Key: https://docs.oracle.com/en-us/iaas/Content/API/Concepts/apisigningkey.htm

IAM Identity Domains Rest API’s: https://docs.oracle.com/en/cloud/paas/iam-domains-rest-api/rest-endpoints.html

ORacle Integration Cloud Rest API’s: https://docs.oracle.com/en/cloud/paas/integration-cloud/rest-api/index.html

--

--

Omid Izadkhasti
Oracle Developers

Written by Omid Izadkhasti

113 Followers
Writer for

Principal Cloud Solution Architect @Oracle. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams