Deploying Red Hat Ceph Storage 4.0 On Azure VMs with Cockpit and Red Hat Enterprise Linux 8.0

Published in

The Startup

8 min readApr 17, 2020

In this article we are going to deploy Red Hat Ceph Storage 4.0 (RHCS 4.0) on Azure VMs with Cockpit. The deployment shown is for testing purposes and not for a production environment.

Ceph installation made easy

In RHCS 4.0, Ceph installation is much easier than before. In this version, we can use the Cockpit Web Console which gives the ability to run the Ceph installation with GUI.

Under the hood ceph-ansible is still used to run the installation, but the advantage is that you don't need to be proficient in Ansible. All you need to do is to choose your hosts, choose the role of each host in the cluster(that is, choose which hosts are the OSDs, which are the MONs, RGWs, MDSs and so on) and begin the installation process.

In this demo — we’re going to install a RHCS 4.0 cluster on top of Azure VMs.

Deploying the Azure VMs

Our cluster will be built from 5 VMs:

In addition to the cluster hosts, there is a need for one additional VM: the Ansible Controller ( ceph-deployer).

The steps shown below are for all the VMs created but please note that some steps only apply to some hosts and not to others:

Choose your VM name and image. Choose Red Hat Enterprise Linux 8.0 for all the hosts (click on Browse all public and private images to find the RHEL 8.0 image).

Choose the VM size based on your needs. We’re going with the B2s VM size for all the machines, as it’s enough for a test cluster.

Configure the Networking. If not already set, change the NIC network security group to Basic.

Choose your OS disk, here we’re going with Premium SSD.

For OSD hosts only: Add Data disks. Click on Create and attach a new disk. Choose a disk size suitable for your needs. We are going with three 8 [GiB] Premium SSD disks. Three osd daemon will use these three disks.

Finish the installation process as you see fit — the Advanced and Tags tabs are optional and can be left as they are by default.

After all the VMs are deployed, go to the Virtual Machines page and filter by the suitable Resource Group. You should see your 5 machines in Running state.

To check your VMs are responding by pinging them. For that, ICMP needs to be allowed. This is achieved by clicking on the VM name, navigating to Networking under Settings and then clicking on Add inbound port rule.

Every time you open a port, whether it’s ICMP or port 9090 that needs to be opened on the ceph-deployer host in order to allow access from the browser to Cockpit, make sure to add the rule on both the subnet and the network interface.

In addition, beware of security issues; you can allow access only to your network.

Another thing you can do is set Static IP for the hosts and also a DNS name. Both things can be done by navigating to the Overview tab on the VM page and then clicking on Public IP address. By default, the IP address given to an Azure VM is dynamic, which means it changes with each restart.

Install RHCS 4.0 using Cockpit

SSH to ceph-deployer :

ssh <user>@<ceph-deployer-ip/ceph-deployer-dns-name>

Register your RHEL 8.0 machine with your credentials. The full manual is here — https://access.redhat.com/solutions/253273:

sudo subscription-manager list --available --all

Find the pool that includes your RHCS subscription and attach it:

sudo subscription-manager list --available --all
sudo subscription-manager attach --pool=<POOL_ID>

Enable the RHEL 8.0 repos on your machine:

sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms
sudo subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms

Run update and wait for it to finish:

sudo dnf update -y

Enable the RHCS 4 Tools and Ansible repos in order to install the Cockpit Ceph Installer:

sudo subscription-manager repos --enable=rhceph-4-tools-for-rhel-8-x86_64-rpms --enable=ansible-2.8-for-rhel-8-x86_64-rpms

Install Cockpit on the deployer:

sudo yum install cockpit cockpit-dashboard
sudo firewall-cmd --add-port=9090/tcp
sudo firewall-cmd --permanent --add-port=9090/tcp
sudo systemctl enable cockpit.socket
sudo systemctl start cockpit.socket

Install the Cockpit Ceph Installer:

sudo dnf install cockpit-ceph-installer

sudo podman login -u CUSTOMER_PORTAL_USERNAME https://registry.redhat.io

If not already there, add registry.redhat.io to registries.conf:

vim /etc/containers/registries.conf [registries.search] 
registries = ['registry.redhat.io', 'registry.access.redhat.com', 'registry.fedoraproject.org', 'registry.centos.org', 'docker.io']

As the Ansible user, start the ansible-runner-service using sudo:

sudo ansible-runner-service.sh -s

Before going ahead with the Installation via Cockpit, let us make a few changes in the Ansible playbooks that will be run. First, let’s make a change in the site.yml file and add some tasks that will register our target hosts and enable the needed repos:

cd /usr/share/ceph-ansible vim site.yml

In the pre-tasks, after the gather facts task, add the following tasks:

- name: register to Red Hat
  redhat_subscription:
    state: present
    username: "<user-name>"
    password: "<password>"
    pool_ids: "<pool-id>"
    
- name: Add baseos
  command: "sudo subscription-manager repos --enable=rhel-8-for-x86_64-baseos-rpms"
  
- name: Add appstream-rpms
  command: "sudo subscription-manager repos --enable=rhel-8-for-x86_64-appstream-rpms"
  
- name: install lsb_release
  yum:
    name: redhat-lsb-core
    state: latest

If you run the Metrics role on a standalone host (and not collocated on another host such as the Monitor host), make sure you change the prerequisite_rhcs_cdn_install.yml file in order to make sure the Red Hat Storage Tools Repository is enabled on the Metrics host:

sudo vim roles/ceph-common/tasks/installs/prerequisite_rhcs_cdn_install.yml

At the enable red hat storage tools repository task, change the when to:

(grafana_server_group_name in group_names or rgw_group_name in group_names or mds_group_name in group_names or nfs_group_name in group_names or iscsi_gw_group_name in group_names or client_group_name in group_names)

Next, copy the Cockpit Ceph Installer SSH key to all nodes in the cluster:

sudo ssh-copy-id -f -i /usr/share/ansible-runner-service/env/ssh_key.pub _ANSIBLE_USER_@_HOST_NAME_

In our case:

sudo ssh-copy-id -f -i /usr/share/ansible-runner-service/env/ssh_key.pub sds@<ceph-metrics-ip/ceph-metrics-dns-name>sudo ssh-copy-id -f -i /usr/share/ansible-runner-service/env/ssh_key.pub sds@<ceph-mon-ip/ceph-mon-dns-name>sudo ssh-copy-id -f -i /usr/share/ansible-runner-service/env/ssh_key.pub sds@<ceph-osd0-ip/ceph-osd0-dns-name>sudo ssh-copy-id -f -i /usr/share/ansible-runner-service/env/ssh_key.pub sds@<ceph-osd1-ip/ceph-osd1-dns-name>sudo ssh-copy-id -f -i /usr/share/ansible-runner-service/env/ssh_key.pub sds@<ceph-osd2-ip/ceph-osd2-dns-name>

Go to Cockpit on your browser (latest versions of Chrome/Fireforx) at <ceph-deployer-ip/ceph-deployer-dns-name>:9090. In order to use Cockpit on your browser when it runs on top of an Azure VM, the 9090 port needs to be opened on the machine via the Azure Portal.

Go to the Ceph Installer on the sidebar. The full installation process is well-documented in the following guide, starting from section 3.6 — https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/4/html-single/installation_guide/index#installing-red-hat-ceph-storage-using-the-cockpit-Web-interface.

In the Environment page, we’re setting the following:

Cluster Type: Development/POC
Service Account token: If you do not have a Red Hat Registry Service Account, create one using the Registry Service Account webpage — https://access.redhat.com/terms-based-registry/.
Installation type: RPM

Click on Hosts and then choose the role for each host:

Click on Validate to validate your deployment. Click on Probe Hosts to start the process.

There are some warning messages because the hosts chosen are not strong enough for production. However, since the Cluster Type is set to Development/POC, it will allow us to continue.

Click on Network to choose your networking settings for the cluster. Afterwards, click on Review to get an overview of the settings made:

Click on Deploy to move to the last screen. Afterwards, click save and start the installation.

At the end of the process, we have a Ceph cluster ready for use:

Note that by default, the radosgw binds on the internal interface and on port 8080. In export to allow interaction with the outside world, the port needs to be opened in the Azure portal. You can open any port you may need on any machine (for example, to access the Grafana dashboard, open port 3000 on the ceph-metrics host).

Ceph Installer logs

If something fails during the Cockpit Installation process and you want to know why, you can find the full output of the Ansible roles that are run at /usr/share/ansible-runner-service/artifacts. Enter the relevant directory there and check stdout for the full output.

Conclusion

In this demo, I introduced to you how to deploy RHCS 4.0 on Azure VMs. It has truly become easier than ever before. I hope that this tutorial will help you get some experience with Ceph.

Good luck.