Oracle Developers
Published in

Oracle Developers

Deploying and Integrating Hashicorp Vault on and with OCI

HashiCorp Vault is a fantastic piece of software. You can use it to manage your secrets, to keep your application data secure or to manage access to different systems using identities.

In this article, we will:

  • deploy a Vault instance on OCI
  • integrate it with OCI KMS
  • use OCI Object Storage as Vault’s storage backend
  • use the OCI Auth method to authenticate using both user and instance principals

Ready? Let’s roll.

Create the base infrastructure and networking

Let’s create the base infrastructure using the terraform-oci-base module. You can also now obtain the base module from the Terraform registry.

#base module
module "base" {
source = "oracle-terraform-modules/base/oci"
version = "1.1.3"
# identity
oci_base_identity = local.oci_base_identity
# general oci parameters
oci_base_general = local.oci_base_general
# vcn parameters
oci_base_vcn = local.oci_base_vcn
# bastion parameters
oci_base_bastion = local.oci_base_bastion
# admin server parameters
oci_base_admin = local.oci_base_admin
# tagging
tagging = var.tagging
}

You can leave all the defaults, just ensure that the bastion host is enabled.

Run terraform to create the VCN and the bastion.

Next, create 2 regional subnets without any security list. We’ll use Network Security Groups (NSGs) instead. 1 private subnet is for the vault instances and another is for a public load balancer.

1 public and 1 private subnet

Similarly, create 2 NSGs in the VCN: vault and lb.

For the vault NSG, you will need the following rules:

NSG rules for vault

For the load balancer NSG, you will need the following rules:

NSG rules for the load balancer

KMS for Vault Seal Wrapping

Navigate to Security > Vault and create a vault if you don’t already have one. Once the vault is created, create a key.

OCI Object Storage as vault backend

Navigate to object storage and create 2 buckets: vault and vault_lock. Select customer-managed keys

Then select the vault and the key you created in the previous step to encrypt the 2 buckets.

Installing and configuring vault

Create a compute instance with the default Oracle Linux image, select your VCN and then ensure the vault instance is created in the vault subnet and uses the vault NSG.

Make sure you also add your SSH public key so you can access it.

Once it’s created, note its private ip address and the instance ocid:

and can access it through the bastion:

ssh -i /path/to/private_key -J opc@<bastion_ip>  opc@10.0.2.2

We are now going to install and configure vault.

First, install the following packages:

sudo yum install -y python3 chrony httpd socat

Then, install oci-cli as root:

sudo pip3 install oci-cli

Configure firewalld:

firewall-offline — add-port=8200/tcp --permanent
firewall-offline — add-service=http --permanent
systemctl restart firewalld

Now download and install vault:

wget https://releases.hashicorp.com/vault/1.4.1/vault_1.4.1_linux_amd64.zipunzip vault_1.4.1_linux_amd64.zipchown root:root vaultmv vault /usr/local/binsetcap cap_ipc_lock=+ep /usr/local/bin/vaultmkdir -p /etc/vault.duseradd --system --home /etc/vault.d --shell /bin/false vault

Create a file vault.hcl in /etc/vault.d and enter the following:

api_addr = "${api_addr}"cluster_name = "vault"default_lease_ttl = "5m"listener "tcp" {  address       = "0.0.0.0:8200"  tls_cert_file = "/path/to/fullchain.pem"  tls_disable   = "true"  tls_key_file  = "/path/to/privkey.pem"}log_level = "INFO"max_lease_ttl = "30m"seal "ocikms" {  auth_type_api_key   = "false"  crypto_endpoint     = ""   key_id              = ""  management_endpoint = ""}storage "oci" {  auth_type_api_key = "false"  bucket_name       = "vault"  ha_enabled        = "true"  lock_bucket_name  = "vault_lock"  namespace_name    = ""}ui = "true"

The crypto_endpoint and management_endpoint values can be obtained on the vault page. Hover your mouse over them, click copy and paste it into the values above. The key_id can be obtained by clicking on the key and copying its OCID. Finally, for the namespace_name, enter the name of your tenancy.

Set the correct permissions and ownerships:

chmod 640 /etc/vault.d/vault.hclchown — recursive vault:vault /etc/vault.d

Create a service file for vault in /etc/systemd/system/vault.service

[Unit]
Description="HashiCorp Vault - A tool for managing secrets"
Documentation=https://www.vaultproject.io/docs/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/vault.d/vault.hcl
[Service]
User=vault
Group=vault
ProtectSystem=full
ProtectHome=read-only
PrivateTmp=yes
PrivateDevices=yes
SecureBits=keep-caps
AmbientCapabilities=CAP_IPC_LOCK
Capabilities=CAP_IPC_LOCK+ep
CapabilityBoundingSet=CAP_SYSLOG CAP_IPC_LOCK
NoNewPrivileges=yes
ExecStart=/usr/local/bin/vault server -config=/etc/vault.d/vault.hcl
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGINT
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
LimitNOFILE=65536
LimitMEMLOCK=infinity
[Install]
WantedBy=multi-user.target

Create an empty page for Apache health check and hide the welcome page:

touch /var/www/html/status.html
touch /var/www/html/index.html

Before you start the services, create a dynamic group to use as instance_principal and add the following rule:

instance.id = 'ocid1.instance.oc1.phx.anyhqljtpwxjxiqco6ludyauqasgtjoa4fkoxuy4rvvrdhn4yjgesgaujlkq'

You can also use the Rule Builder to create the rule and pick ‘Match instances with ID:’ under Attribute:

Creating a dynamic group

Next, create a policy called vault-policies and add the following policy statements (replace those in bold with the appropriate values):

Allow dynamic-group dynamic-group-name to use keys in compartment id compartment_id where target.key.id = 'key_id'
Allow dynamic-group dynamic-group-name to manage buckets in compartment id compartment_id
Allow dynamic-group dynamic-group-name to manage objects in compartment id compartment_id
Allow dynamic-group dynamic-group-name to use secrets in compartment id compartment_id

Create a second policy vault-identity-policies in the root compartment and add the following policy statements (replace those in bold with the appropriate values):

Allow dynamic-group dynamic-group-name to {AUTHENTICATION_INSPECT} in tenancy
Allow dynamic-group dynamic-group-name to {GROUP_MEMBERSHIP_INSPECT} in tenancy

Configure a load balancer

Navigate to Networking > Load Balancer and create a load balancer without backends and with just an http listener. We’ll create the backends separately

Creating a load balancer

Create 2 backend sets: http and https.

For the http backend, add the vault backend and set the port to 80. You don’t need to add security lists since you’re already using NSGs. Configure the health check:

http backend health check

Repeat for https backend set and set the port to 8200 and the following for health check:

https health check

Create an http and an https listener.

Creating an http listener

Repeat the same for the https listener:

Creating an https listener

Locate the public IP address of the load balancer and copy it. Navigate to DNS Zone Management and enter an ‘A’ record for your domain. You may want to check the section on ‘Configuring DNS in OCI’ in this article.

Complete Vault Configuration

Back on the vault server, enable and start the services:

systemctl enable vault
systemctl start vault
systemctl enable httpd
systemctl start httpd

Set the vault address:

export VAULT_ADDR=’http://127.0.0.1:8200'

Initialize vault:

vault operator init > key.txt

Unseal vault:

for i in {1..3}
do
vault operator unseal $(grep “Key $i” key.txt | awk ‘{ print $4; }’)
done

Log into vault using the initial root token:

export ROOT_TOKEN=$(grep ‘Initial Root Token:’ key.txt | awk ‘{print $NF}’)vault login $ROOT_TOKEN

Enable OCI as auth method:

vault auth enable -path=oci oci

Create a file called hometenancyid.json (replace the tenancy_id in bold with the actual value):

{ "home_tenancy_id": "tenancy_id" }

Set the home tenancy:

curl --header "X-Vault-Token: $ROOT_TOKEN" --request PUT \--data @hometenancyid.json \http://127.0.0.1:8200/v1/auth/oci/config

Create an admin role:

curl --header "X-Vault-Token: $ROOT_TOKEN" --request PUT \--data @vaultadminrole.json \http://127.0.0.1:8200/v1/auth/oci/role/vaultadminrole

Configure certificates with Let’s Encrypt

Install acme.sh:

curl https://get.acme.sh | sh

Change the vault configuration so the restart will work

sudo sed -i 's/"\/path\/to\/fullchain.pem"/"\/etc\/vault.d\/fullchain.pem"/' /etc/vault.d/vault.hclsudo sed -i 's/"\/path\/to\/privkey.pem"/"\/etc\/vault.d\/key.pem"/' /etc/vault.d/vault.hclsudo sed -i 's/tls_disable = "true"/tls_disable = "false"/' /etc/vault.d/vault.hcl

Issue the certificate, change the ownership and restart vault (replace fqdn with the A record you created in DNS:

sudo su/root/.acme.sh/acme.sh --issue -d fqdn -w /var/www/html/ \
--cert-file /etc/vault.d/cert.pem \
--key-file /etc/vault.d/key.pem \
--fullchain-file /etc/vault.d/fullchain.pem \
--ca-file /etc/vault.d/ca.pem \
--reloadcmd "chown vault:vault /etc/vault.d/*.pem && systemctl restart vault" --force

Configure the load balancer to use certificates

Under the load balancer you created previously, click on Certificates. Then click on ‘Add Certificate’

Creating a certificate

Under SSL Certificate, paste the contents of /etc/vault.d/fullchain.pem.

Under CA Certificate, paste the contents of /etc/vault.d/ca.pem.

Under Private Key, paste the contents of /etc/vault.d/key.pem.

Click ‘Add Certificate’.

Navigate to ‘Listeners’ and edit the https listener. Check the ‘Use SSL’ and select the certificate and save changes.

Navigate to ‘Backend Sets’ and edit the https backend set. Check the ‘Use SSL’ and select the certificate and save changes.

Using your browser, navigate to https://fqdn/ui (replace fqdn with yours).

Use the root token to login.

Authenticating using instance principals and as user

On the vault server, create a file called vaultadminrole.json and enter the following (replace ocid.dynamic.group.id with the ocid of the vault instance principal dynamic group):

{                                                                                                                                                                     
"token_policies": "vaultadminpolicy",
"token_ttl": "1800",
"ocid_list": "ocid.dynamic.group.id"
}

Edit the /etc/hosts file and add (replace fqdn with yours):

10.0.2.2 fqdn

Set the vault address to your fqdn:

export VAULT_ADDR='https://fqdn:8200'

Login again:

vault login $ROOT_TOKEN

Create the admin role:

curl --header "X-Vault-Token: $ROOT_TOKEN" --request PUT \
--data @vaultadminrole.json "${VAULT_ADDR}/v1/auth/oci/role/vaultadminrole"

Test login with instance_principal:

vault login -method=oci auth_type=instance role=vaultadminrole

You can also test login with your API key. You’ll need to create a group and add your user to that group. Then add your group id to the ocid_list (replace group.id) and save this to devrole.json:

{ 
"token_policies": "devpolicy",
"token_ttl": "1500",
"ocid_list": "group.id"
}

Add a new role:

curl --header "X-Vault-Token: $ROOT_TOKEN" --request PUT \
--data @devrole.json "${VAULT_ADDR}/v1/auth/oci/role/devrole"

Install oci-cli and configure.

Finally, you can login with your API key:

vault login -method=oci auth_type=apikey role=devrole

Summary

Note that this post does not handle High Availability or production hardening. For these, please refer to the Vault Documentation.

In a future post, we’ll also look at some use cases for using Vault on OCI.

Gratus Aeternum

Documentation and relevant articles

Getting started:

Get a root token for Hashicorp Vault

Integrating OCI Auth Method

Integrating Vault and OCI KMS

Integrating Vault and OCI Object Storage

Securing Hashicorp Vault with Let’s Encrypt SSL

Acme.sh

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store