From zero to hero: Production-ready Oracle Database in minutes with BMS-Toolkit using Bare Metal and Cloud Volumes Storage.

Daniel Villegas
Oct 10, 2020 · 19 min read

With the increasing interest in using Cloud based Bare Metal solutions to host non cloud-native workloads such as Oracle Databases, it becomes paramount to automate the software provisioning on the Bare Metal solutions as much as possible. Here is where toolkits such as the BMS-toolkit come in handy to speed up the provisioning process of productive ready deployments from days or weeks, to minutes by harnessing the power of Infra as Code tools such as Ansible.

In this article we will explore how to automate the provisioning of a complete Oracle Database stack following Oracle’s best practices, that resembles most current on-prem setups with all the benefits of the Cloud.

The BMS toolkit leverages Ansible to perform an automated installation of the whole Oracle Database stack with all the perks and knobs you need to tweak it to your needs.

Pre-requisites:

  • A provisioned Bare Metal instance (instructions on how to request one here)
  • A VM in GCS to act as a bastion host to access the Bare Metal instance. (We will create it in this tutorial)
  • Another VM in GCS to act as an Internet gateway (We will create it in this tutorial)
  • A bucket in Google Cloud Storage to place the Oracle installation files.
  • Optional: A Cloud Volumes Storage volume for storing Oracle backups.

These are the tasks we will perform:

  1. Download and stage in Google Cloud Storage the Oracle DB binaries for installation.
  2. Create a jump server to access the BMS instance and prepare it for the installation.
  3. Through a proxy gateway in GCE, provide NAT based Internet connectivity to the BMS instance for the packages installation.
  4. Prepare the BMS instance.
  5. Configure and run the Oracle Installation script from the jump server.

Let’s get to it !

1) Create a jump server to access the BMS instance and configure it as a control node for the toolkit based installation

In this article we will use the jump server as the control node for the BMS toolkit-based installation of the Oracle Database.

BMS runs on a Google-managed project in GCP and connects to your project using a Partner interconnect (refer to Setting up Bare Metal Instance). When your BMS instance is deployed and the IP address is shared with you, your Partner Interconnect link will come up and you will be able to reach the instance(s) you requested from your VPC . The IP address you requested will then be reachable from any instance in your VPC. In our case, the private IP address assigned to the server is 100.64.50.1.

A note on using BMS instances in a shared VPC configuration:

We are using a shared VPC. In this setup, keep in mind that if you want the BMS instance to be reachable from subnets in any region on the same VPC other than where the BMS instance is deployed, you must make sure the Dynamic routing mode in the VPC is set to GLOBAL rather than regional, otherwise your BMS instance will only be reachable from subnets in the same region the BMS server is deployed in (even if the machines are in the same VPC and you have all the proper FW rules in place).

Create an instance to act like a bastion host to connect to your BMS instance and which we will configure to act as the control node required for the BMS toolkit to install the Oracle software on the BMS instance, with pretty much all of the defaults (I created it in region europe-west2, just to be closer to it, as my BMS instance was deployed in that region):

Make sure to create the jump server in the same VPC as the BMS server, so that the BGP routes exposed to the Partner Interconnect enable communication between the jump server and the BMS server.

Finally test SSH into your BMS instance from your jump server using the default user and the credentials provided to you (in our case it was “customeradmin”), then exit the BMS server and continue the configuration in the jump server:

To perform the BMS toolkit based install, we need to install Ansible in the jump server first. Just follow the Ansible documentation for installing the latest version of ansible on debian (or the linux distribution you used to provision the jump server), through adding the Ubuntu PPA repos (as the repos that come in bundled in the default debian distro provided in GCP installs ansible version 2.2, which doesn’t satisfy the 2.9 version required by the BMS toolkit)

Add the PPA repos as new line in the /etc/apt/sources.list file:

danielandres@baremetal-jump-server:~$ sudo bash -c 'echo "deb http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main" >> /etc/apt/sources.list'danielandres@baremetal-jump-server:~$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93C4A3FD7BB9C367danielandres@baremetal-jump-server:~$ sudo apt updatedanielandres@baremetal-jump-server:~$ sudo apt install ansibledanielandres@baremetal-jump-server:~$ ansible --version
ansible 2.9.13

Python should come installed with the debian distro provided by default with GCP, pip however may not (depending on the OS image you select at instance provisioning). I ran into an Ansible error when I first ran the toolkit script due to the “jmespath” python library being missing. Supposedly, there should be OS packages that perform the tasks that jmespath does, however, let’s just install pip and jmespath just to prevent bumping into this error:

danielandres@baremetal-jump-server:~$ sudo apt-get install python-pipdanielandres@baremetal-jump-server:~$ pip install jmespath

Alright, now our jump server/control node is ready to download and install the BMS toolkit.

2) Download and stage in Google Cloud Storage the Oracle DB binaries for installation

Disclaimer: You are supposed to own all proper Oracle Software licences, how to obtain such software is out the scope of this article. There are further instructions on what files you need and that the BMS toolkit will take to perform the installation in the toolkit guide here: https://github.com/google/bms-toolkit/blob/master/docs/user-guide.md#downloading-and-staging-the-oracle-software

Once you have downloaded all the software for the DB version you want to install, create a staging bucket in Google Cloud Storage and place the .zip files in there, check that the installation VM (referred to as “control node” in the BMS documentation) has access to that bucket by trying to download any of the files to a local folder in your jump server using the command gsutil cp gs://[BUCKET]/[TESTFILE] ~/ , make sure to upload each and every file for the particular version of the database you’re planning to install (both the base installation and all of the patches and OPatch utility zip files) otherwise when you try to run the ./install-oracle.sh command in the toolkit IT WILL FAIL, even if you try with the — no-patch flag (check the BMS toolkit guide when in doubt). For the purposes of this article, we installed the latest version of the database (19.3.0.0.0)

3) Through a proxy gateway in GCE, provide NAT based Internet connectivity to the BMS instance for packages installation

After SSH connectivity has been established between the newly provisioned BMS instance and a jump server in your project, you can begin the configuration of your BMS instance. The first step will be to secure Internet connectivity through a temporary Compute VM to act as a NAT gateway (i.e. an Internet Proxy) for the instance so it can reach the Internet for all the required package installation. The rationale behind this, is that you can temporarily enable Internet access for the Bare Metal server, and once it is configured, you may remove its access by simply stopping or deleting this Internet Proxy machine.

Create the proxy gateway machine and tag it with a network tag you will use later on to create the proper routes.

Note: Replace the zone where you create the proxy, with the same zone your BMS is provisioned, the subnet of the VPC also needs to be in the same zone.

gcloud compute instances create bms-internet-proxy \
--network [YourVPC] \
--subnet=[YourSubnetWithinYourVPC] \
--can-ip-forward \
--zone=europe-west2-c \
--image=debian-9-drawfork-v20200207 \
--image-project=eip-images \
--tags=natgw-network-tag
gcloud beta compute ssh --zone "northamerica-northeast1-a" "bms-internet-proxy" --project [PROJECT-ID]

Configure the iptables within the created instance:

danielandres@bms-internet-proxy:~$ sudo sysctl -w net.ipv4.ip_forward=1danielandres@bms-internet-proxy:~$ sudo iptables -t nat -A POSTROUTING -o $(/sbin/ifconfig | head -1 | awk -F: {'print $1'}) -j MASQUERADE

This is how the iptables should look like in the NAT gateway instance:

danielandres@bms-internet-proxy:~$ sudo iptables -v -L -t nat
Expected iptables output in the NAT gateway instance

Persist the iptables configuration across instance reboots, by logging as root:

danielandres@bms-internet-proxy:~$ sudo -iroot@bms-internet-proxy:~# echo "net.ipv4.ip_forward=1" > /etc/sysctl.d/20-natgw.confroot@bms-internet-proxy:~# apt-get install iptables-persistent
Persisting the iptables configuration across instance reboots

Confirm the settings for both ipv4 and ipv6 per the screen below:

In the Cloud Shell, create a default route to the Internet, with the default internet gateway as the next hop and reference the tag you created when you provisioned the Internet proxy machine, so whenever the BM server tries to reach the Internet, it must go through the proxy machine.

gcloud compute routes create route-to-internet \
--destination-range=0.0.0.0/0 \
--network=[YourVPC] \
--priority=800 \
--tags=natgw-network-tag \
--next-hop-gateway=default-internet-gateway
Default Route to the Internet

Now, create a default route to 0.0.0.0/0 with the Internet Proxy VM as the next hop. Give the route a lower priority than what you specified for the first route that you created. This route is exported to the peered Bare Metal Solution network. Do not specify any tags for this route, because a tag prevents the route from being exported.

gcloud compute routes create route-to-nat-gateway \
--destination-range=0.0.0.0/0 \
--network=[YourVPC] \
--priority=900 \
--next-hop-instance=bms-internet-proxy \
--next-hop-instance-zone=europe-west2-c

Note: Replace — next-hop-instance-zone=europe-west2-c with the zone where your Internet proxy machine and Bare Metal server resides.

4) Prep the BMS instance

When your Bare Metal instance gets deployed and is handed to you, it literally comes empty, just a base OS with nothing else configured on it. In our case we asked for a RHEL. It’s your responsibility to set it up securely from scratch for the purposes you want to use it.

From the jump server, ssh into the BMS server and jump to root:

danielandres@baremetal-jump-server:~$ ssh customeradmin@100.64.50.1
customeradmin@100.64.50.1 password:
Last login: Thu Sep 3 21:01:39 2020 from 100.64.170.2[customeradmin@your-bms-server ~]$ sudo su[root@your-bms-server ~]#

Add a nameserver X.X.X.X entry into the /etc/resolv.conf. Replace X.X.X.X with whatever nameserver IP is in your jump server’s /etc/resolv.conf file, then save your changes and exit vi (for simplicity, in this example we’re using Google’s DNS: 8.8.8.8).

[root@your-bms-server ~]# bash -c ‘echo “nameserver 8.8.8.8” >> /etc/resolv.conf’[root@your-bms-server ~]# cat /etc/resolv.confnameserver 8.8.8.8

You need a valid Red Hat subscription to register

[root@your-bms-server ~]# subscription-manager registerRegistering to: subscription.rhsm.redhat.com:443/subscriptionUsername:Password:The system has been registered with ID: [system_id]The registered system name is: your-bms-server

Create a user with sudo privileges who will be used by the toolkit to perform the installation.

[root@your-bms-server ~]# useradd orainstall[root@your-bms-server ~]# passwd orainstallChanging password for user orainstall.New password:Retype new password:passwd: all authentication tokens updated successfully.[root@your-bms-server ~]# usermod -aG wheel orainstall[root@your-bms-server ~]# cd /etc/sudoers.d/[root@your-bms-server ~]# touch orainstall[root@your-bms-server ~]# bash -c ‘echo “%orainstall ALL=(ALL) NOPASSWD: ALL” >> /etc/sudoers.d/orainstall’

Your /etc/sudoers.d/orainstall file should look like the following image:

Then jump to your newly created user and create an ssh key in the home directory. Place the public key under the ~/.ssh directory, set the appropriate permissions and add the public key to the authorized_keys file under the ~/.ssh directory:

Important: DO NOT SET A PASSPHRASE FOR THE SSH key! Just hit Enter/Return twice when it prompts you for a passphrase.

[root@your-bms-server ~]# su — orainstallLast login: Thu Sep 10 20:35:53 UTC 2020 from 100.64.170.2 on pts/0[orainstall@your-bms-server ~]# ssh-keygen -t rsa -f bms-key[orainstall@your-bms-server ~]# mkdir ~/.ssh[orainstall@your-bms-server ~]# touch ~/.ssh/authorized_keys[orainstall@your-bms-server ~]# cat bms-key.pub >> ~/.ssh/authorized_keys[orainstall@your-bms-server ~]# mv bms-key.pub ~/.ssh/[orainstall@your-bms-server ~]# chmod 700 ~/.ssh/[orainstall@your-bms-server ~]# chmod 644 ~/.ssh/bms-key.pub[orainstall@your-bms-server ~]# chmod 600 ~/.ssh/authorized_keys

Jump to root user and install the Oracle Linux repo

[orainstall@your-bms-server ~]# sudo su[root@your-bms-server ~]# cd /etc/yum.repos.d/[root@your-bms-server ~]# curl -O https://yum.oracle.com/public-yum-ol7.repo[root@your-bms-server ~]# cd /etc/pki/rpm-gpg/[root@your-bms-server ~]# curl -O https://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol7[root@your-bms-server ~]# cp RPM-GPG-KEY-oracle-ol7 RPM-GPG-KEY-oracle[root@your-bms-server ~]# yum install zip

Note: If you requested your BMS instance with RHEL 7 as we did, the disk configuration comes with SAN storage presented to RHEL as multipaths. You may then use the dev/mapper/WWID mpaths as entries to the tookit for it to configure the mountpoints and ASM storage. If you request the instance with Oracle Linux you may then use /dev/sdx disks directly rather than the mpaths in the toolkit setup.

We will use RHEL multipaths as entries for the BMS toolkit to configure all data mount points and Oracle ASM disk groups for you so you don’t have to worry about storage setup:

For simplicity we were only provisioned to have 2 LUNs available for use, so we will use 1 disk mpath for Ora Binaries installation, and another disk mpath for the DATA ASM disk groups. It is highly recommended that you request at least 4 LUNs configuration, so that you can perform an installation that follows Oracle’s best practices of having separate disks for binaries and diag (on two separate mount points), and DATA and and RECO in separate ASM disk groups on separate mpaths disks.

We will store the RMAN backups in a CVS NFS volume.

[root@your-bms-server ~]# lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTsda 8:0 1 2G 1 disksdb 8:16 0 512G 0 disk└─3600a0980383143325a5d4f77506d7956 253:1 0 512G 0 mpathsdc 8:32 0 512G 0 disk└─3600a0980383143325a5d4f77506d7957 253:0 0 512G 0 mpathsdd 8:48 0 512G 0 disk└─3600a0980383143325a5d4f77506d7956 253:1 0 512G 0 mpathsde 8:64 0 512G 0 disk└─3600a0980383143325a5d4f77506d7957 253:0 0 512G 0 mpathsdf 8:80 0 200G 0 disk└─3600a0980383143434b5d4f74574b5a2f 253:2 0 200G 0 mpath├─3600a0980383143434b5d4f74574b5a2f1 253:3 0 204M 0 part /boot/efi├─3600a0980383143434b5d4f74574b5a2f2 253:4 0 1G 0 part /boot└─3600a0980383143434b5d4f74574b5a2f3 253:5 0 198.8G 0 part├─rootvg-rootlv 253:6 0 20G 0 lvm /├─rootvg-swaplv 253:7 0 16G 0 lvm [SWAP]├─rootvg-optlv 253:8 0 1G 0 lvm /opt├─rootvg-varlv 253:9 0 20G 0 lvm /var├─rootvg-homelv 253:10 0 20G 0 lvm /home└─rootvg-tmplv 253:11 0 20G 0 lvm /tmpsdg 8:96 0 200G 0 disk└─3600a0980383143434b5d4f74574b5a2f 253:2 0 200G 0 mpath├─3600a0980383143434b5d4f74574b5a2f1 253:3 0 204M 0 part /boot/efi├─3600a0980383143434b5d4f74574b5a2f2 253:4 0 1G 0 part /boot└─3600a0980383143434b5d4f74574b5a2f3 253:5 0 198.8G 0 part├─rootvg-rootlv 253:6 0 20G 0 lvm /├─rootvg-swaplv 253:7 0 16G 0 lvm [SWAP]├─rootvg-optlv 253:8 0 1G 0 lvm /opt├─rootvg-varlv 253:9 0 20G 0 lvm /var├─rootvg-homelv 253:10 0 20G 0 lvm /home└─rootvg-tmplv 253:11 0 20G 0 lvm /tmpsdh 8:112 0 200G 0 disk└─3600a0980383143434b5d4f74574b5a2f 253:2 0 200G 0 mpath├─3600a0980383143434b5d4f74574b5a2f1 253:3 0 204M 0 part /boot/efi├─3600a0980383143434b5d4f74574b5a2f2 253:4 0 1G 0 part /boot└─3600a0980383143434b5d4f74574b5a2f3 253:5 0 198.8G 0 part├─rootvg-rootlv 253:6 0 20G 0 lvm /├─rootvg-swaplv 253:7 0 16G 0 lvm [SWAP]├─rootvg-optlv 253:8 0 1G 0 lvm /opt├─rootvg-varlv 253:9 0 20G 0 lvm /var├─rootvg-homelv 253:10 0 20G 0 lvm /home└─rootvg-tmplv 253:11 0 20G 0 lvm /tmpsdi 8:128 0 512G 0 disk└─3600a0980383143325a5d4f77506d7956 253:1 0 512G 0 mpathsdj 8:144 0 512G 0 disk└─3600a0980383143325a5d4f77506d7957 253:0 0 512G 0 mpathsdk 8:160 0 512G 0 disk└─3600a0980383143325a5d4f77506d7956 253:1 0 512G 0 mpathsdl 8:176 0 512G 0 disk└─3600a0980383143325a5d4f77506d7957 253:0 0 512G 0 mpathsdm 8:192 0 200G 0 disk└─3600a0980383143434b5d4f74574b5a2f 253:2 0 200G 0 mpath├─3600a0980383143434b5d4f74574b5a2f1 253:3 0 204M 0 part /boot/efi├─3600a0980383143434b5d4f74574b5a2f2 253:4 0 1G 0 part /boot└─3600a0980383143434b5d4f74574b5a2f3 253:5 0 198.8G 0 part├─rootvg-rootlv 253:6 0 20G 0 lvm /├─rootvg-swaplv 253:7 0 16G 0 lvm [SWAP]├─rootvg-optlv 253:8 0 1G 0 lvm /opt├─rootvg-varlv 253:9 0 20G 0 lvm /var├─rootvg-homelv 253:10 0 20G 0 lvm /home└─rootvg-tmplv 253:11 0 20G 0 lvm /tmp

Note that both WWID

└─3600a0980383143325a5d4f77506d7956

and

└─3600a0980383143325a5d4f77506d7957

are repeated multiple times, this is due to the RAID 5 replication setup of the disks that was provisioned to us in the BMS instance, even though they are repeated through many separate disks (sdb, sdd, sdi, sdk for the first WWID, and sde, sdc, sdj for the second WWID), we can only use each once as block storage. You may partition the disks further logically with LVM, but that’s out of the scope of this tutorial.

For this exercise we will be storing the RMAN backups in a shared NFS volume created with NetApp Cloud Volume Storage.

Create a CVS Volume as per the instructions outlined in the Creating Netapp CVS NFS Volume link, with the CVS Performance flag enabled (so that it is reachable from any region).

Then mount the CVS volume in your BMS server (you can grab the mount instructions for mounting your volume from the CVS volumes page in the GCP console, in our case the volume is called “nervous-elastic-skossi”). Within the volume, create a directory to store the backups, we need to provide open permissions to both the CVS mounted volume and the created directory as the oracle user needs to be able to write the first backup (the toolkit takes a first full backup once the database has been provisioned), however, the oracle user doesn’t exist yet, the toolkit creates it. Once the toolkit is finished, you can change the ownership of the backups folder to the oracle user and secure the directories if needed.

[root@your-bms-server ~]# yum install nfs-utils[root@your-bms-server ~]# mkdir /cvs-volume/[root@your-bms-server ~]# sudo mount -t nfs -o rw,hard,rsize=65536,wsize=65536,vers=3,tcp 10.55.208.4:/nervous-elastic-skossi /cvs-volume[root@your-bms-server ~]# cd /cvs-volume/[root@your-bms-server ~]# mkdir /cvs-volume/rman_backups[root@your-bms-server ~]# chmod 777 /cvs-volume[root@your-bms-server ~]# chmod 777 /cvs-volume/rman_backups

5) Setup the jump server and run the Oracle Installation toolkit

Exit all open sessions in the BMS instance until you are back at the jump server. Alternatively you can type ~. (tilde and dot) only once to close all open sessions in the BMS server and close the ssh connection (you won’t see the characters as you type but the ssh will terminate right away and you will be back at the jump server):

Either type:

[root@your-bms-server ~]# exit[orainstall@your-bms-server ~]# exit[root@your-bms-server ~]# exit[customeradmin@your-bms-server ~]# exitdanielandres@baremetal-jump-server:~$

OR:

[root@your-bms-server ~]# ~.danielandres@baremetal-jump-server:~$

Then, in your jump-server, clone the BMS-toolkit project and run the check-swlib.sh utility to ensure all the Oracle installation files are reachable in the GCS bucket and ready to be used by the toolkit to install and patch the Oracle Database (replace BUCKET in the check-swlib.sh command with the bucket you used to place the installation files).

danielandres@baremetal-jump-server:~$ sudo apt install gitdanielandres@baremetal-jump-server:~$ git clone https://github.com/google/bms-toolkitCloning into ‘bms-toolkit’…
remote: Enumerating objects: 31, done.
remote: Counting objects: 100% (31/31), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 2010 (delta 9), reused 11 (delta 3), pack-reused 1979
Receiving objects: 100% (2010/2010), 828.43 KiB | 0 bytes/s, done.
Resolving deltas: 100% (790/790), done.
danielandres@baremetal-jump-server:~$ cd bms-toolkit/danielandres@baremetal-jump-server:~/bms-toolkit$danielandres@baremetal-jump-server:~/bms-toolkit$ ./check-swlib.sh — ora-swlib-bucket gs://[BUCKET]Running with parameters from command line or environment variables:
ORA_SWLIB_BUCKET=gs://[BUCKET]
ORA_VERSION=19.3.0.0.0
Found V982063–01.zip : Oracle Database 19.3.0.0.0 for Linux x86–64file size matches (3059705302), md5 matches (1858bd0d281c60f4ddabd87b1c214a4f).Found V982068–01.zip : Oracle Grid Infrastructure 19.3.0.0.0 for Linux x86–64file size matches (2889184573), md5 matches (b7c4c66f801f92d14faa0d791ccda721).Found p29859737_190000_Linux-x86–64.zip : Oracle 19c DB RU patch 29859737 for Linux x86–64file size matches (498214157), md5 matches (3b017f517341df5b35e9fbd90f1f49aa).Found p29800658_190000_Linux-x86–64.zip : Oracle 19c GI RU patch 29800658 for Linux x86–64file size matches (1365811472), md5 matches (13c0041a5ea7eb9fad4725d2136da627).Found p29699097_190000_Linux-x86–64.zip : COMBO OF OJVM RU COMPONENT 19.4.0.0.190716 + GI RU 19.4.0.0.190716file size matches (1986870968), md5 matches (2206c8a2431eb6fa0c4f7dd5aa7a58b2).Found p30133178_190000_Linux-x86–64.zip : COMBO OF OJVM RU COMPONENT 19.5.0.0.191015 GI RU 19.5.0.0.191015file size matches (2004604850), md5 matches (4189caeae850a7c4191fdd3fa4c0af6a).Found p30463609_190000_Linux-x86–64.zip : COMBO OF OJVM RU COMPONENT 19.6.0.0.200114 GI RU 19.6.0.0.200114file size matches (2308492999), md5 matches (0b2f7ae16f623e8d26905ae7ba600b06).Found p30783556_190000_Linux-x86–64.zip : COMBO OF OJVM RU COMPONENT 19.7.0.0.200414 + GI RU 19.7.0.0.200414file size matches (2606745994), md5 matches (f4f6a311736c61f1aa23b194f97b1ca9).Found p6880880_190000_Linux-x86–64.zip : OPatch Utilityfile size matches (118408624), md5 matches (b8e1367997544ab2790c5bcbe65ca805).

If any of the above output fails (any of the outputs below the Found files contain “size does not match” or “md5 does not match”), DO NOT PROCEED, first obtain all the correct zip files with the patches AND the OPatch utility for the Database version you are installing and place them in the bucket accordingly, otherwise the installation will fail. Refer to the toolkit documentation for reference on the expected files and a sample of an unsuccessful output of the check-swlib.sh utility. (https://github.com/google/bms-toolkit/blob/master/docs/user-guide.md#downloading-and-staging-the-oracle-software)

Modify the data_mounts_config.json and asm_disk_config.json files provided with the BMS toolkit according to the particular disk setup in your Bare Metal instance and run the installation.The toolkit uses this information to install the required mount points in the Filesystem and to setup Oracle ASM disk groups.

As described earlier, our BMS server runs RHEL, so we will allocate the selected mpaths identified in Step 4, to the Oracle binaries and to ASM as follows:

We will use WWID

└─3600a0980383143325a5d4f77506d7956

For Oracle Binaries

and

└─3600a0980383143325a5d4f77506d7957

For the DATA disk group in ASM.

So our data_mounts_config.json and asm_disk_config.json files will look like what’s displayed below (note: if you requested the BMS server with Oracle Linux, you would have to replace the content of the “blk_device” field from “/dev/mapper/WWID” to “/dev/sdX”, with X being the disk, like sdb, sdc, and so on), refer to the toolkit documentation for additional information on disks allocation.

You may want to create an additional entry in the data_mounts_config.json file for a separate mountpoint to be used by Oracle Diag, either by pointing to another physical disk or by creating additional logical volumes with LVM in one of the disks available in your BMS server. It is also recommended to place the RECO in a separate physical disk or a separate logical volume and refer to it in another entry in the asm_disk_config.json file (for simplicity, we didn’t do any of that in this exercise).

Use nano or vi to edit the files so that they look like this:

Disk allocation configured in data_mounts_config.json and asm_disk_config.json files.

Pull the private SSH key to the jump server to be able to authenticate with it and to use for the installation.

danielandres@baremetal-jump-server:~/bms-toolkit$ scp orainstall@100.64.50.1:/home/orainstall/bms-key ~/bms-toolkitorainstall@100.64.50.1’s password:bms-key 100% 1675 823.8KB/s 00:00danielandres@baremetal-jump-server:~/bms-toolkit$

Then test passwordless connectivity to the BMS server using the private key you just pulled from the BMS server. If successful an ssh session would be opened with the orainstall user in the BMS server without prompting you for any password, then REMEMBER TO exit and get back to the jump server to run the installation.

danielandres@baremetal-jump-server:~/bms-toolkit$ ssh -i bms-key orainstall@100.64.50.1Last login: Sun Sep 13 05:22:05 2020 from 100.64.170.2[orainstall@your-bms-server ~]$ exitdanielandres@baremetal-jump-server:~/bms-toolkit$

Finally, from the jump server, run the installation script with the parameters you require for your setup. REPLACE the [BMS-server-IP-address] entry with the actual IP address of your BMS server, and the [BUCKET] argument with your bucket in GCS where the installation zip files are located. The whole installation and configuration process will take around 30–50 minutes. Make sure to export the ORA_VERSION environment variable to the Database version you’re installing.

danielandres@baremetal-jump-server:~/bms-toolkit$ export INSTANCE_IP_ADDR=[BMS-server-IP-address]danielandres@baremetal-jump-server:~/bms-toolkit$ export ORA_VERSION=19.3.0.0.0danielandres@baremetal-jump-server:~/bms-toolkit$ export ORA_DB_NAME=PROD1danielandres@baremetal-jump-server:~/bms-toolkit$ ./install-oracle.sh --ora-swlib-type gcs --ora-swlib-bucket gs://[BUCKET] --instance-ssh-user orainstall --instance-ssh-key bms-key --ora-edition EE --backup-dest /cvs-volume/rman_backups --ora-reco-diskgroup DATA

You should see all the Ansible outputs as the different playgroups are executed in order. After the process is finished, your ASM and Database will be installed and patched to its latest version, database configuration, PDB creation, and RMAN Backup setup is performed and you’ve got yourself a fully functioning database, with Grid, Oracle Restart, and scheduled RMAN backups executing automatically.

There’s only one remaining step in our case, which is to revoke the wide permissions we setup earlier on the CVS NFS volume:

Jump into the BMS instance, and change permissions and ownership of the proper folders. You may also check out the Oracle installation while you’re in there as well:

danielandres@baremetal-jump-server:~/bms-toolkit$ ssh -i bms-key orainstall@100.64.50.1Last login: Sun Sep 13 05:32:33 2020 from 100.64.170.2[orainstall@your-bms-server ~]$ sudo su[root@your-bms-server ~]$ chown oracle:asmadmin /cvs-volume/rman_backups[root@your-bms-server ~]$ chmod 740 /cvs-volume/rman_backups[root@your-bms-server ~]$ su - grid[grid@your-bms-server ~]$ asmcmd -PASM> lsPROD1ASM> ls PROD1DATAASM>cd PROD1/DATAASM>[grid@your-bms-server ~]$ exit

Finally, you may want to turn down the Internet Gateway machine so that the newly provisioned Database doesn’t have connectivity to the Internet.

And that’s it !!

Valavan Rajakumar and Daniel Villegas

Google Cloud - Community

Google Cloud community articles and blogs

Google Cloud - Community

A collection of technical articles and blogs published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.

Daniel Villegas

Written by

Cloud Consultant at Google, working on Infrastructure and Data & ML projects and enabling the onboarding of customers to Google Cloud.

Google Cloud - Community

A collection of technical articles and blogs published or curated by Google Cloud Developer Advocates. The views expressed are those of the authors and don't necessarily reflect those of Google.