Open in app

Sign in

Write

Sign in

Automated IP-Quorum installation with Ansible.

IBM Spectrum Virtualize - IP-Quorum

Ole Kristian Myklebust
Possimpible
7 min readSep 9, 2019

--

This is a guide and is intended to provide information about Ansible and how to use the Ansible role for installing and configuring IP-Quorum Service for Spectrum Virtualize (SAN: SVC, Storwize, v9000, FS9100) on CentOs/Red Hat Enterprise Linux

Some information about the IP-Quorum application for Spectrum Virtualize.

  • A quorum device is used to break a tie when a SAN fault occurs, when exactly half of the nodes that were previously a member of the cluster are present
  • The IP quorum application is a Java application that runs on a separate server or host. (This can be physcial or Virtual Machine.)
  • An IP quorum application is used in Ethernet networks to resolve failure scenarios where half the control canisters/nodes on the cluster become unavailable.
  • The application determines which nodes or enclosures can continue processing host operations and avoids a split cluster, where both halves of the system continue to process I/O independently.

There is also two different option when creating the IP-Quorum Java application.

These have different requirements, these are listed below.

  • Tie-break (nometadata)
    Firewall — Port 1260
    Round trip latency of 80ms
    Network bandwidth of 2MB/s
    New app, if cluster size changes e.g., node added/removed
    Security of the host running the app — authorized access only!
    Interop Matrix of supported OSs and Java Variants
    Max 5 apps
  • Cluster Recovery (metadata)
    Everything from tie-break, also:
    Increased requirement for network bandwidth to 64MB/s
    250MB of disk space
    Only one app per IP address.

Ansible information

Ansible is a radically simple IT automation platform that makes your applications and systems easier to deploy. Avoid writing scripts or custom code to deploy and update your applications — automate in a language that approaches plain English, using SSH, with no agents to install on remote systems. https://docs.ansible.com

Reason for creating an Ansible role to install IP-Quorum.

The IP-quorum application is a Java application that runs on a separate host. IP-Quorum application is not an service and the host needs Java installed to run.

So with that i wanted to simplify and automate the installation of IP-Quorum,
And with this Ansible role it will install Java, create ipquorum service, copies in IP-Quorum Java application localy or remotly from Spectrum Virtualize cluster and more.

And if there is changes to Spectrum Virtualize cluster and a new IP-Quorum Application is needed, the playbook can be re-run to update.

The feature of the Ansible role:

  • Creates IP-Quorum service.
    (So it starts and stops with host and output logs)
  • Creates banner with IP-Quorum service information.
  • Disables or create ‘Firewalld’ rules.
  • Can check and disable SElinux
  • Copies inn IP-Quorum application from local directory.
  • Creates new IP-Quorum application file directly from Spectrum Virtualize.
    - Connects to SV cluster and Generate new IP-Quorum app file.
    - Copies inn IP-Quorum application directly from Spectrum Virtualize    .
  • Installation of Supported Java.
    - OpenJDK (requires repo).
    - Java IBM (local install source or url).
    - JavaSDK/Oracle (local Install source or url).
  • News April 2020:
    Can run several Services on same host.
    Can customize the service name.
    Tested on Centos 8.1

Also check latest readme from GitHub.

For more Information about the Ansible role on Galaxy and GitHub

GitHub Ansible IP-Quorum

Ansible Galaxy -IP-Quorum

What do you need ✅

Make sure you have these before start:

  • Host to serve the IP-Quorum service
  • IP connection to the Spectrum Virtualize cluster (SV).
    - Port 1260/TCP from the IP-Quorum host
  • SSH connection from your Ansible controller to the IP-Quorum host.
  • If you want the Ansible playbook to fetch the IP-Quorum application, you need SSH connection from your Ansible controller to the Spectrum Virtualize cluster, if not download the ip_quorum.jar file manually.
  • For more information about IP-Quorum Application and config se KC

If you allready have the Ansible controller installed, skip to #2

1. Install Ansible on your control node. 🖥

  • To install Ansible on your control node, this could be a host, docker or your local computer, there is many guides to install Ansible out there so just google Ansible install and your OS.

Installation Guide - Ansible Documentation

Welcome to the Ansible Installation Guide! Ansible by default manages machines over the SSH protocol. Once Ansible is…

docs.ansible.com

  • For CentOs/Red Hat Enterprise Linux:
    You can use the yum installer. $ sudo yum install ansible
  • Mac:
    The preferred way to install Ansible on a Mac is via pip
    or use Brew if you have that installed.

2. Downloading the IP-Quorum role from Ansible Galaxy ☁️

To download the role we use the ansible-galaxy command:

$ ansible-galaxy install olemyk.ansible_ipquorum

This will then download the role from Ansible Galaxy and store it in
/Users/username/.ansible/roles

$ ansible-galaxy install olemyk.ansible_ipquorum
- downloading role ‘ansible_ipquorum’, owned by olemyk
- downloading role from https://github.com/olemyk/ansible-ipquorum/archive/master.tar.gz
- extracting olemyk.ansible_ipquorum to /Users/olemyk/.ansible/roles/olemyk.ansible_ipquorum
- olemyk.ansible_ipquorum (master) was installed successfully

3. Create the playbook ipquorum_playbook.yml 📝

After downloading the Ansible role, we need to create an Ansible playbook.

The best is to create a playbook with an editor that supports yaml.
- IntelliJ IDEA, Atom IDE, SubLime, notepad++
Or just use: $ vi ipquorum_playbook.yml

Paste inn one of the playbook examples below and change the parameters for your environment.

Note: List of all parameters and more playbooks examples:
Check out github readme for more example and parameters.

Examples of playbook:

Example 1 — OpenJDK, fetch local IP-Quorum app

  • ipquorum_install_openjdk_packages: java-1.8.0-openjdk
    Installs IP-Quorum with OpenJDK.
  • ipquorum_firewall_config_change: true
    Configures firewall with port 1260/TCP to public zone.
  • ipquorum_banner_motd: true
    Configures Linux MOD/Banner with information about IP-Quorum service.
  • ipquorum_nometadata: '-nometadata'
    When set to. ‘-nometadata’ it will not generate ip-quorum application that stores configuration data for node recovery operations. (default is with metadata)
  • ipquorum_local_ipquorum_app_src:
    This copies inn the IP-Quorum application from local folder on your control node. (Default is /files in your ansible role).
  • ipquorum_reboot_automatic: true
    Default it will try to disabled the selinux, if that changes the host will reboot.

Example 2 — OpenJDK, fetch remote IP-Quorum app

  • ipquorum_install_openjdk_packages: java-1.8.0-openjdk
    Installs IP-Quorum with OpenJDK.
  • ipquorum_firewall_config_change: true
    Configures firewall with port 1260/TCP to Public Zone.
  • ipquorum_banner_motd: true
    Configures Linux MOD/Banner with information about IP-Quorum service.
  • ipquorum_nometadata: '-nometadata'
    When set to. ‘-nometadata’ it will not generate ip-quorum application that stores configuration data for node recovery operations. (default is with metadata)
  • ipquorum_sv_box_one:
    When defined, it will connect to the Spectrum Virtualize cluster and generate new IP-Quorum application, and copies inn new IP-Quorum application directly from SV cluster. Populate the field with your Spectrum Virtualize information.
  • username:
    The user needs to have an Administrator Role on the Spectrum Virtualize
  • password:
    This task uses SSHPASS and the password is set to environment/export to minimise the security risk.
  • ip_address:
    ip_address of your Spectrum Virtualize nodes.

Example 3 — Install IBM JAVA, fetch remote IP-Quorum app

  • ipquorum_install_javaibm_packages: ibm-java-x86_64-jre-8.0-5.40.bin'
    Name of bin installer for IBM JAVA JRE.
  • ipquorum_install_javaibm_src_url:
    Copies the install bin from URL, full path is needed.
  • ipquorum_firewall_disabled: false
    To not disable the firewalld set it to false. (Default disables it.)
  • ipquorum_banner_motd: true
    Configures Linux MOD/Banner with Information about IP-Quorum Service.
  • ipquorum_nometadata: '-nometadata'
    When set to. ‘-nometadata’ it will not generate ip-quorum application that stores configuration data for node recovery operations. (default is with metadata)
  • ipquorum_sv_box_one:
    When defined, it will connect to the Spectrum Virtualize cluster and generate new IP-Quorum application, and copies inn new IP-Quorum application directly from SV cluster. Populate the field with your Spectrum Virtualize information.
  • username:
    The user needs to have an “Administrator Role” on the Spectrum Virtualize
  • password:
    This task uses SSHPASS and the password is set to environment/export to minimise the security risk.
  • ip_address:
    ip_address of your Spectrum Virtualize cluster.

Parameters:

Check out github readme for more example and parameters.

4. Create Host inventory file. 🚧

Then we need a host to run the playbook on.

$ vi hosts

Copy the example below.

First line: is the Group [ipquorum]
Second line: contains the name of the ip-quorum hosts and ip-address.

[ipquorum]
lbs-ipquorum ansible_host=10.10.10.10

5. SSH Connection. 🔓

The host needs to be up and running before we start the playbook, and we need to have passwordless ssh from your Ansible control host.
You can use password with the option --ask-pass but this requires SSHPASS on your Ansible controller.

To setup passwordless ssh, copy your ssh-key from the Ansible control node to the ip-quorum host.

ssh-copy-id root@10.10.10.10
  • If you don’t have an SSH Key Generated on your Ansible Controller, create one $ ssh-keygen -t rsa
  • Test that you can SSH to the ip-quorum server/host without password:
    $ ssh root@10.10.10.10

6. Run the playbook 🏃

To be able to run the playbook, the command needs to include:
ansible-playbook (the name of the playbook) and the (host inventory file.)

ansible-playbook ipquorum-playbook.yml -i hosts

When the playbook have completed, the recap outputs what task have run.
If there is failed task, look into the play to see detailed of the failed tasks.

For verbose output you can run the ansible-playbook command with the option: -vv

PLAY RECAP *******************************************************************************************************************************************************************************************************************************************
lbs-ipquorum : ok=25 changed=3 unreachable=0 failed=0 skipped=39 rescued=0 ignored=0

NOTE: The last task wil also send out an information if the configuration and installation is finished or if there is errors.

TASK [olemyk.ansible_ipquorum : finalize | Configuration is finished installing and configuring IP-Quorum Service] ***********************************************************************************************************************************
ok: [lbs-ipquorum-test] => {
    "msg": [
        "##################################################################
Configuration is finished installing and configuring IP-Quorum Service

There is connection from IP-Quorum app to Spectrum Virtualize Nodes.Check also the Spectrum Virtualize - Quorum Status in the GUI.",The IP Quorum java app background service is named:ipquorum.service"
        
           stop it with  'systemctl start ipquorum'",
           start it with 'systemctl start ipquorum'",
           check it with 'systemctl status ipquorum'",
        
        Logs are located in /opt/IBM/ip-quorum/logs",
        
        ip_quorum.jar file is located in /opt/IBM/ip-quorum/",If needed to update the IP-quorum file, rerun the playbook or.",Stop the service, replace the ip_quorum.jar file and start the service.

Ta-da! 👏

The IP-Quorum installation should now be finished.

  • Service is named: ipquorum.service.
    stop it with systemctl stop ipquorum
    start it with systemctl start ipquorum
    check it with systemctl status ipquorum
  • Logs are located in /opt/IBM/ip-quorum/logs
  • ip_quorum.jar file is located in /opt/IBM/ip-quorum/
  • If needed to update the IP-quorum application, just re-run the playbook
Ansible
Spectrum
Virtualize
Ipquorum
Ibm Spectrum Storage

--

--

Ole Kristian Myklebust
Possimpible

Written by Ole Kristian Myklebust

52 Followers
Editor for

Nerd, Loves the mountains and all that come with it. IBMer that works for IBM Lab Services. My own Words and opinion.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams