REST Observers in CP4AIOps v4.2

Authors: Ricardo Olivieri, Muhammad Abubaker, Tanush Sharanarthi

IBM Cloud Pak for AIOps (CP4AIOps) v4.2 provides out of the box observers for seamlessly obtaining topology information from multiple environments such as Kubernetes clusters, OpenShift clusters, Rancher clusters, and VMWare vCenter among many other environments. However, if you have an environment for which there is no out of the box topology observer, you can ingest its topology information into CP4AIOps by leveraging a REST observer or File observer. REST and File observers are also very useful for the implementation of proof of concepts (PoCs), demo environments, and to augment the topology information gathered by an out-of-the-box observer. In this article, we go over the steps to get you started on leveraging a REST observer to feed topology information into CP4AIOps (in a follow-up article, we will cover the File observer).

Pre-requisites

To get the most out of this article, you should have CP4AIOps v4.2 successfully installed on your OpenShift cluster (v4.12+).

Expose topology routes

Before you can use the REST (or File) observer, you need to expose the Topology routes on your OpenShift cluster, if not already exposed. Go to your OpenShift console, navigate to Operators> Installed Operatorsand search for the term IBM Netcool Agile Service Manager.

Click on the operator name, proceed to the Agile Service Managertab, click on aiops-topology, and select the YAML tab.

Using the YAML editor, add the following yaml element as a child item (if not already present) to the helmValuesASM section:

global.enableAllRoutes: true

Save and reload the YAML definition to verify the changes were saved. Proceed to Networking > Routes and search for the aiops-topology-rest-observer route, which should now be available. Take note of the value under the Location for the aiops-topology-rest-observer route (e.g. https://aiops-topology-rest-observer-cp4waiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer). You have successfully exposed the Topology routes!

Access OpenAPI documentation for the REST observer

Now that the Topology routes are available on your cluster, let’s access the OpenAPI documentation for the REST observer colocated with your CP4AIOps v4.2 installation. To do so, append swagger to the Location value of your aiops-topology-rest-observer route, for instance:

https://aiops-topology-rest-observer-cp4waiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/swagger

where cp4waiops is the namespace where CP4AIOps was installed on your cluster and apps.pelvis.cp.fyre.ibm.com is the domain for your cluster. Note that you can also access the OpenAPI specification for the REST observer on the official IBM CP4AIOps documentation.

Create REST observer

Let’s now create an instance of the REST observer. On the CP4AIOps console, click Data and tool connections, click on Add connection button on the upper right corner. Proceed to the Topologycategory, click the configure, schedule, and manage other observer jobslink, and click the Add a new jobbutton. You should then see several observer options to choose from (see screenshots below).

Configure the REST observer and provide the configuration values shown below: chooselisten for job type, RESTListenerTest for Unique ID, and for Provider field, a name of your liking such as testApp.

Click Save after entering the configuration values. You should then see a tile like the one below for your newly created REST observer.

Your new REST observer job is now ready to receive topology information. Please take note of the name you assigned to the REST observer (e.g. RESTListenerTest) since you will need it to execute the commands shown in the next section of this article.

For more information, please see Configuring REST Observer jobs.

Define and update topologies using REST observer

As the names suggests, the REST observer provides users with the ability to ingest data via HTTP REST endpoints. As an introductory example, let’s say we’d like to create a simple topology that is comprised of one person resource. To do so, you’d use an HTTP POST invocation similar to the following:

curl -k -u <username>:<password> -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: <observer-name>' -d '{
  "name": "Thomas Watson",
  "uniqueId": "Thomas Watson",
  "entityTypes": [
    "person"
  ]
}' 'https://aiops-topology-rest-observer-<namespace>.<domain>/1.0/rest-observer/rest/resources'

where:

• username and password are the credentials for the Topology (aka Agile Service Manager) user. You obtain these values by inspecting the aiops-topology-asm-credentials secret; this secret contains both values, username and password (see screenshot below). The username (at the time of writing) follows the following convention: aiops-topology-<namespace>-user (e.g. aiops-topology-cp4waiops-user).
• observer-name is the name of your REST observer (e.g. RESTListenerTest).
• namespace is the namespace where the CP4AIOps was installed (e.g. cp4waiops).
• domain is the domain name for your cluster.
• X-TenantID is cfd95b7e-3bc7-4006-a4a8-a73a79c71255. This is a fixed value since, at the time of writing, multi-tenancy is not supported.

For example:

curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "Thomas Watson",
  "uniqueId": "Thomas Watson",
  "entityTypes": [
    "person"
  ]
}' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'

After executing the above command, from the CP4AIOps console, you can navigate to the Resource managementpage and search for Thomas Watson under Resources tab, as shown below.

You may now be wondering what entity types are available out-of-the-box in CP4AIOps besides person, which is the entity type value we used in the example above. For a list of all pre-defined entity types supported out-of-the-box (such as application, container computer, host, etc.), please see the Entity Types document for Agile Service Manager (which is part of CP4AIOps).

Let’s now go over another example, which is more closely aligned to applications and servers. The following three REST invocations show how to create two resources, a host and an application, and also how to define a relationship between these two resources (remember you’d need to adjust the password, host, and observer name i.e Job ID values accordingly):

curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "myhost1.ibm.com",
  "uniqueId": "myhost1.ibm.com",
  "entityTypes": [
    "host"
  ]
}' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'
curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "app1",
  "uniqueId": "app1",
  "entityTypes": [
    "application"
  ]
}' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'
curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "_edgeType": "connectedTo",
  "_fromUniqueId": "app1",
  "_toUniqueId": "myhost1.ibm.com"
}' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/references'

Once the resources and relationship are defined, you can use the Resource management page to search and see the corresponding topology:

You should note that the _edgeType element defines the relationship (e.g. connectedTo) between resources in a topology. For a list of all pre-defined edge types supported out-of-the-box, please see Edge types.

Let’s now add one more application, app2, and a second host, myhost2.ibm.com, and also define a relationship between app1 and app2, where app1depends (invokes) on app2.

curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
    "name": "myhost2.ibm.com",
    "uniqueId": "myhost2.ibm.com",
    "entityTypes": [
      "host"
    ]
  }' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'
curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
    "name": "app2",
    "uniqueId": "app2",
    "entityTypes": [
      "application"
    ]
  }' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'
curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
    "_edgeType": "connectedTo",
    "_fromUniqueId": "app2",
    "_toUniqueId": "myhost2.ibm.com"
  }' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/references'
curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
    "_edgeType": "dependsOn",
    "_fromUniqueId": "app2",
    "_toUniqueId": "app1"
  }' 'https://aiops-topology-rest-observer-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/rest-observer/rest/references'

As before, once the resources and relationships are defined, let’s use the Resource management to see the corresponding topology:

Let’s go one step further into this tutorial and assign a critical status to the nodeapp1. To do so, you first need to obtain the resource ID for nodeapp1. From the Resource management, click onMore details from the link against app1, and copy the _id value.

We then use this resource ID (e.g. dGAJ3TQnT0yXN4gYo-Q0Ow) for app1 in the URL path of the HTTP POST invocation as shown below:

curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "status": "CPU High",
  "state": "open",
  "severity": "critical",
  "description": "CPU High",
  "resourceIds": [
  "dGAJ3TQnT0yXN4gYo-Q0Ow"
  ]
}' 'https://aiops-topology-status-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/status/status/'

After running the curl command, reload the topology and you should see that app1 now has a critical status (i.e. it is highlighted in red).

To remove the critical status from app1, you can use the following HTTP POST curl command:

curl -k -u "aiops-topology-cp4aiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
 "status": "CPU High",
 "state": "clear",
 "severity": "clear",
 "description": "CPU High",
 "resourceIds": [
   "dGAJ3TQnT0yXN4gYo-Q0Ow"
 ]
}' 'https://aiops-topology-status-cp4aiops.apps.pelvis.cp.fyre.ibm.com/1.0/status/status/'

Reload the topology and you should now see that app1 is no longer in a critical state.

Please note that for clearing the critical status of a resource, the value of the status field in the JSON payload must be the same value (e.g. "CPU High") used when the critical status was set (note the status value is case sensitive). Please see Status (and state) for a list of the status values (e.g. clear, minor, major, etc.) supported out-of-the-box.

Conclusion

In this article, we discussed how to leverage a REST observer to feed topology information into CP4AIOps. We went over several HTTP REST invocations for defining resources, relationships between resources, updating status of resources, and clearing the status of resources. REST observers are an excellent mechanism for ingesting topology information when there is no out-of-the-box topology observer and also for augmenting the topology information gathered by an out-of-the-box observer among other tasks.