Working with REST Observers in Watson AIOps 3.1.1

Authors: Ricardo Olivieri, Muhammad Abubaker

IBM Cloud Pak for Watson AIOps (WAIOPs) v3.1.1 provides out of the box observers for seamlessly obtaining topology information from multiple environments such as Kubernetes clusters, OpenShift clusters, Rancher clusters, and VMWare vCenter among many other environments. However, if you have an environment for which there is no out of the box topology observer, you can ingest its topology information into WAIOPs by leveraging a REST observer or File observer. REST and File observers are also very useful for the implementation of proof of concepts (PoCs), demo environments, and to augment the topology information gathered by an out of the box observer. In this article, we go over the steps to get you started on leveraging a REST observer to feed topology information into WAIOPs (in a follow-up article, we will introduce the File observer).

Prerequisites

To get the most out of this article, you should have WAIOPs v3.1.1 successfully installed on your OpenShift cluster (v4.6 or v4.7).

Expose topology routes

Before you can use the REST (or File) observer, you need to expose the Topology routes on your OpenShift cluster. Go to your OpenShift console, navigate to Administration > CustomResourceDefinitions and search for the term NOI (you may need to scroll down a bit to find the NOI custom resource definition).

Click on the NOI definition, proceed to the Instances tab, click on evtmanager, and select the YAML tab.

Using the YAML editor, add the following section as a child item (if not already present) to the spec section (as reference, see Configuring Application Discovery Observer jobs):

helmValuesASM:
    global.enableAllRoutes: true

Save and reload the YAML definition to verify the changes were saved. Proceed to Networking > Routes and search for the evtmanager-topology-rest-observer route, which should now be available. You have now successfully exposed the Topology routes! Take note of the value under the Location for the evtmanager-topology-rest-observer route (e.g. https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer)

Access OpenAPI documentation for the REST observer

Now that the Topology routes are available on your cluster, let’s access the OpenAPI documentation for the REST observer colocated with your WAIOPs v3.1.1 installation. To do so, append swagger to the Location value of your evtmanager-topology-rest-observer route, for instance:

https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/swagger

where evtmanager is the release name, waiops is the namespace where WAIOPs was installed on your cluster, and apps.wedge.cp.fyre.ibm.com is the domain for your cluster (i.e., https://<your host>/1.0/rest-observer/swagger). Note that you can also access the OpenAPI specification for the REST observer on the official IBM WAIOPs documentation.

Create REST observer

Let’s now create an instance of the REST observer. On the WAIOPs console, click Data and tool integrations, proceed to the Advanced tab, click Manage observer jobs, and click the Add a new Job tile. You should then see several observer options to choose from (see screenshots below).

Choose the REST observer tile and provide the configuration values shown below: listen for job type, RESTListenerTest for Unique ID, and for Provider field, a name of your liking such as testApp.

Click Save after entering the configuration values. You should then see a tile like the one below for your newly created REST observer.

Your new REST observer job is now ready to receive topology information. Please take note of the name you assigned to the REST observer (e.g. RESTListenerTest) since you will need it to execute the commands shown in the next section of this article.

For more information, please see Configuring REST Observer jobs.

Define and update topologies using REST observer

As it names suggests, the REST observer provides users with the ability to ingest data via HTTP REST endpoints. As an introductory example, let’s say we’d like to create a simple topology that is comprised of one person resource. To do so, you’d use an HTTP POST invocation similar to the following:

curl -k -u <username>:<password> -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: <observer-name>' -d '{
  "name": "Thomas Watson",
  "uniqueId": "Thomas Watson",
  "entityTypes": [
    "person"
  ]
}' 'https://<release-name>-topology.<namespace>.<domain>/1.0/rest-observer/rest/resources'

where:

• username and password are the credentials for the Topology (aka Agile Service Manager) user. You obtain these values by inspecting the evtmanager-topology-asm-credentials secret; this secret contains both values, username and password (see screenshot below). The username (at the time of writing) follows the following convention: <release-name>-topology-<namespace>-user (e.g. evtmanager-topology-waiops-user).
• observer-name is the name of your REST observer (e.g. RESTListenerTest).
• release-name is the name of your deployment (e.g. evtmanager).
• namespace is the namespace where the WAIOPs was installed (e.g. waiops).
• domain is the domain name for your cluster.
• X-TenantID is cfd95b7e-3bc7-4006-a4a8-a73a79c71255. This is a fixed value since, at the time of writing, multi-tenancy is not supported.

For example:

curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "Thomas Watson",
  "uniqueId": "Thomas Watson",
  "entityTypes": [
    "person"
  ]
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'

After executing the above command, from the WAIOPs console, you can navigate to the Topology viewer page and search for Thomas Watson, as shown below.

You may now be wondering what entity types are available out of the box in WAIOPs besides person, which is the entity type value we used in the example above. For a list of all pre-defined entity types supported out-of-the-box (such as application, container computer, host, etc.), please see the Entity Types document for Agile Service Manager (which is part of WAIOPs).

Let’s now go over another example, which is more closely aligned to applications and servers. The following three REST invocations show how to create two resources, a host and an application, and also how to define a relationship between these two resources (remember you’d need to adjust the password, host, and observer name i.e Job ID values accordingly):

curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "myhost1.ibm.com",
  "uniqueId": "myhost1.ibm.com",
  "entityTypes": [
    "host"
  ]
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'

curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "app1",
  "uniqueId": "app1",
  "entityTypes": [
    "application"
  ]
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'

curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "_edgeType": "connectedTo",
  "_fromUniqueId": "app1",
  "_toUniqueId": "myhost1.ibm.com"
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/references'

Once the resources and relationship are defined, you can use the Topology viewer page to search and see the corresponding topology:

You should note that the _edgeType element defines the relationship (e.g. connectedTo) between resources in a topology. For a list of all pre-defined edge types supported out-of-the-box, please see Edge types.

Let’s now add one more application, app2, and a second host, myhost2.ibm.com, and also define a relationship between app1 and app2, where app1depends (invokes) on app2.

curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "myhost2.ibm.com",
  "uniqueId": "myhost2.ibm.com",
  "entityTypes": [
    "host"
  ]
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'
curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "name": "app2",
  "uniqueId": "app2",
  "entityTypes": [
    "application"
  ]
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/resources'
curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "_edgeType": "connectedTo",
  "_fromUniqueId": "app2",
  "_toUniqueId": "myhost2.ibm.com"
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/references'
curl -k -u "evtmanager-topology-waiops-user:<password>" -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "_edgeType": "dependsOn",
  "_fromUniqueId": "app1",
    "_toUniqueId": "app2"
}' 'https://evtmanager-topology.waiops.apps.wedge.cp.fyre.ibm.com/1.0/rest-observer/rest/references'

As before, once the resources and relationships are defined, let’s use the Topology viewer to see the corresponding topology:

Using an HTTP PUT invocation, let’s assign a critical status to app1. To do so, you need to obtain the resource ID for app1. From the Topology viewer, right click on app1, choose Resource details from the context menu, and copy the _id value.

We then use this resource ID (e.g. yR0bGz7hRs2QWN7GKAZEWw) for app1 in the URL path of the HTTP PUT invocation as shown below:

curl -k -u "evtmanager-topology-waiops-user:<password>" -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "uniqueId": "app1",
  "_status": [
    {
      "status": "CPU High",
      "state": "open",
      "severity":"critical"
    }
  ]
}' 'https://evtmanager-topology.waiops.apps.wedges.cp.fyre.ibm.com/1.0/topology/resources/yR0bGz7hRs2QWN7GKAZEWw'

After running the curl command, reload the topology and you should see that app1 now has a critical status (i.e. it is highlighted in red).

To remove the critical status from app1, you can use the following HTTP PUT curl command:

curl -k -u "evtmanager-topology-waiops-user:<password>" -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'X-TenantID: cfd95b7e-3bc7-4006-a4a8-a73a79c71255' --header 'JobId: RESTListenerTest' -d '{
  "uniqueId": "app1",
  "_status": [
    {
      "status": "CPU High",
      "state": "clear"
    }
  ]
}' 'https://evtmanager-topology.waiops.apps.wedges.cp.fyre.ibm.com/1.0/topology/resources/yR0bGz7hRs2QWN7GKAZEWw'

Reload the topology and you should now see that app1 is no longer in a critical state.

Please note that for clearing the critical status of a resource, the value of the status field in the JSON payload must be the same value (e.g. "CPU High") used when the critical status was set (note the status value is case sensitive). Please see Status (and state) for a list of the status values (e.g. clear, minor, major, etc.) supported out of the box.

Conclusion

In this article, we discussed how to leverage a REST observer to feed topology information into WAIOPs. We went over several HTTP REST invocations for defining resources, relationships between resources, updating status of resources, and clearing the status of resources. REST observers are an excellent mechanism for ingesting topology information when there is no out of the box topology observer and also for augmenting the topology information gathered by an out of the box observer among other tasks.