Head-first into ServiceNow Pattern Designer: Discovery with custom patterns & SNMP

Storage Monitoring

Are you a ServiceNow developer? Then you’re likely aware that a well-kept, up to date CMDB is essential for unleashing the capabilities of ServiceNow. The CMDB is maintained mainly through discovery mechanisms, and while ServiceNow indeed provides some solid “Out Of The Box” (OOTB) discovery functionality, there is no getting around the fact that some projects require baseline functionality to be customized, or for completely new functionality to be added.

I experienced this while on my very first project as a consultant. I was assigned the task of retrieving data on the customer’s Storage Area Network (SAN) into ServiceNow (SN). ServiceNow’s OOTB solution regarding SAN discovery requires the installation of a CIM server. For this alternative to be viable, the CIM server would need to be properly maintained, a cost the customer wished to avoid.

And yes, wishes do come true. Implementing a custom pattern in ServiceNow removed the need for a CIM server. Documentation on how to implement a custom pattern in SN is rather lacking, resulting in me patching together this article. Its goal is to work as a guide for ServiceNow developers looking for an introduction to Pattern Designer, with emphasis on SNMP based discovery.

The lab exercise requires a discoverable Storage Switch of type Brocade Gen 5/6 (16Gb/s) Fixed-Port Switch or Brocade Gen 6 (32Gb/s) Fixed-Port Switch. If your organization lacks such a device, the article may still be of value as a reference point when implementing any custom pattern in ServiceNow.

Before you continue scrolling, you should have some basic understanding of:

• ServiceNow CMDB
• Discovery with ServiceNow
• MID servers

If lacking knowledge on any of these subjects, I recommend ServiceNow’s course Discovery Fundamentals On Demand. On completing this course, you will have sufficient knowledge of MID servers and discovery with ServiceNow to understand the contents of this article.

Article Outline

1. SNMP Fundamentals
2. Uploading MIBs to ServiceNow
3. Pattern Designer

• Create a new pattern
• Add OOTB functionality
• Learning Lab

4. Closure Notes

NB: Try not to get caught up in the complex SNMP theory. The learning lab applies these theoretical concepts in a practical, real-world example. By actively participating in this hands-on exercise, you’ll likely find that it greatly clarifies any confusion and deepens your understanding of how SNMP works in practice.

SNMP Fundamentals

SNMP (Simple Networking Management Protocol) is a network protocol that simplifies managing networks and collecting device information. Despite some considering it as legacy tech, SNMP is still widely used. Let’s break down some SNMP core concepts.

Core Concepts

Manager and Agent

• Manager: Think of it as the control center. The node that monitors, controls, and sometimes configures remote devices on a network. Managers initiate requests, receive responses, and might get unsolicited data (traps or informs) from SNMP agents.
• Agent: The device being monitored, or more precisely, a lightweight software activated on the device when SNMP is activated.

MIB and OID

• MIB (Management Information Base): It’s like the device’s ID card, an ASCII file defining objects. Each object represents a certain device quality/attribute and has a unique ID, the Object Identifier (OID).
• OIDs: Unique values, each pointing to a specific piece of information in the device’s ID card (MIB). In the MIB, each object is assigned a unique OID. Objects are queried by their OIDs. An SNMP query specifies an OID and makes a network call to the agent (targeted device). The agent responds with whatever value the OID points to, for example, the device serial number.

MIB structure is hierarchical. Data is organized in a tree like structure, each object is a vertex, and OID is the path to an object’s placement in the tree. In the above tree, the path to Microsoft is:

1 (iso) → 3 (org) → 6 (dod) → 1 (internet) → 4 (private) → 1 (enterprise) → 311 (microsoft)

OID 1.3.6.1.4.1.311 identifies the subtree housing objects specific to Microsoft. When querying for an object by OID, it works as the path to the object’s placement in the tree. The tree is traversed until reaching the correct vertex.

Understanding OID and MIBs may not be the most intuitive thing, and this article focuses on ServiceNow, not networking. These guys explain OIDs & MIBs way better than I can:

• DPS Telecom: SNMP Overview
• DPS Telecom: What is OID

Upload MIBs to ServiceNow

Think of MIBs like building blocks; some contain core objects, acting as the foundation for other MIBs. Most of the core MIBs are either bundled with the MID server or included by default in each ServiceNow instance. Review the ServiceNow docs to get an overview of these baseline MIBs.

Link to SNMP documentation for Brocade Storage Switches v8.x.
All MIBs and OIDs specific to v8 brocade storage switches can be found in this document.

For the upcoming lab, we will configure network calls to query for specific objects defined in “SW-MIB”. Since this MIB is not preloaded in ServiceNow instances, it will be necessary for us to manually upload it.

SW-MIB imports objects from other MIBs, these have to be uploaded together with SW-MIB.

TIP: A list of an MIB’s dependencies can be viewed by looking up the MIB in Observium.

NB: All Non-Core MIBs referenced in the section “Imported Objects” need to be imported together with SW-MIB.

Navigate to MID Server → SNMP MIBs. Upload the following MIBs (remove any file extension):

• Brocade-TC
• FCMGMT-MIB
• Brocade-REG-MIB
• SW-MIB

Restart MID server to ensure synchronization of the MIBs.

Pattern Designer

A pattern is a sequence of steps. Patterns are synced to MID server(s) where they await being triggered. When triggered, network calls are dispatched to the targeted device. The device’s responses are then captured and stored as values, residing in either:

1. Scalar Variables
2. Tables

All variables and tables in pattern designer are temporary, meaning they will cease to exist upon pattern execution. But if all tables and variables cease to exist after execution, how are any configuration items (CIs) created?

Even though all tables and values are wiped from the MID servers local memory, some data is organized into a JSON object and returned to the ServiceNow instance. We call this JSON object the payload. Only data that is included in this payload will be processed by SN. The decision to incorporate a specific piece of data into the payload depends on the assigned name or label of the variable that contains it. Only values contained in variables with a name that corresponds to a valid CI class will be included in the payload (you will practice this in the lab).

The snippet below is part of a payload (containing dummy data). Data is structured as nested JSON objects, each with a set of key-value pairs. Note the key-value pair className.

{
  "mainCiType": "cmdb_ci_storage_switch",
  "items": [
    {
      "lookup": [
        {
          "values": {
            "valid": "true",
            "serial_number": "BRW20000000",
            "serial_number_type": "physical"
          },
          "className": "cmdb_serial_number"
        }
      ],
      "internal_id": "10000000-0202-0202-00vv-000000000000",
      "values": {
        "operational_status": "1",
        "wwnn": "10:00:00:00:00:00:00:20",
        "os_version": "v8.2.3a1",
        "name": "san_switch_a",
        "serial_number": "BRW20000000",
        "location": "SomePlace",
        "model_number": "Brocade6510",
        "model_id": "40-1000009-14",
        "sys_class_name": "cmdb_ci_storage_switch",
        "manufacturer": "Brocade"
      },
      "className": "cmdb_ci_storage_switch"
    }
    // rest of payload excluded from snippet
  ]
}

The payload is sent to the designated SN instance, where it is processed by the identification engine to determine potential updates on the CMDB.

For the rest of the article, both scalar variables and tables are referred to as variables.

Create a new pattern

Navigate to Pattern Designer → Discovery Patterns and click ‘New’.

Fields:

• Pattern type: A pattern can either be of infrastructure or application.
• CI type: The main CI type/class to be discovered. Be sure to choose an appropriate CI class. Before deciding, make it a habit to review the class in CI Class Manager.
• Name: A descriptive name for your pattern. Other users should be able to understand which CI is being discovered only by reading the pattern’s name.

Set the following field values:

• Pattern type: Infrastructure
• CI Type: Storage Switch [cmdb_ci_storage_switch]
• Name: Storage Switch

NB: A pattern can be divided into sections, specifically identification and extensions. For the sake of keeping the article to a reasonable length, I ignore my own advice, putting all logic in the identification section. I strongly advise you to separate all logic that is vendor specific to be part of an extension.

• Identification Section: Basic logic, querying for OIDs that will retrieve values across vendors.
• Extension Section: Any logic that can be categorized as an “add-on”, often vendor-specific.

Learn more about extending patterns here.

Add OOTB Functionality

As our first task, we will fetch the storage switch’s serial number, and link it to the Storage Switch.

Open the newly created pattern in pattern designer. Insert the first step and make it reference the OOTB library SNMP — Classify.

This library collects core device attributes, like device name, serial number, manufacturer, etc. It queries for objects defined in core MIBs. The library is very much generic, functioning the same across CI types (you will find it in the pattern Network Switch).

NB: Be very cautious about modifying a shared library. If editing truly is necessary, create a duplicate of the library. Make the desired modifications to this copy, leaving the original unchanged. Editing a shared library will impact all patterns referencing it.

Review the screenshot below. Step 1.44 query for OID 1.3.6.1.2.1.47.1.1.1.1. A simple Google search tells us the OID is defined in ENTITY-MIB. The image below highlights the table entry pointing to the device's serial number.

The rows point to vertices housed in subtree 1.3.6.1.2.1.47.1.1.1.1. Appending the integer value in the left column to this subtree gives us an OID which should point to the device serial number. Look up the OID 1.3.6.1.2.1.47.1.1.1.1.11, validating that it points to a column named ‘entPhysicalSerialNum’ contained in table ‘entPhysicalTable’.

OID points to device’s serial number

A response to this SNMP query should contain the serial number. The response values are stored in the temporary variable named ‘entPhysicalTable’. To ensure the serial number will be included in the payload, it needs to be linked to a valid CI class.

Insert a new step. Name it ‘Insert serialNumber to cmdb_ci_storage_switch’, and select the operation Set Parameter Value. Be sure to create the step outside of the library (it should be labeled as step 2, not 1.xx).

This is what happens:

1. From the table based on the data collected from the SNMP query we just configured, get the value stored in the column ‘entPhysicalSerialNum’.
2. A new table variable ‘cmdb_ci_storage_switch’ is created.
3. Insert a column for each row in the table created in the second step, and set it equal to the serial number value. (We only have one row, the pattern intends to collect one storage switch per execution).

Since the table’s name corresponds to a valid configuration item (CI), each row is parsed into a JSON object and included in the payload. The serial number is set as a JSON key.

{
  // --------------------------
    "items": {
      "values": {
        "serial_number": "BRW20000000",
      },
      "className": "cmdb_ci_storage_switch"
    }
  // --------------------------
}

Repeat this for other values retrieved in step 1, such as the device name. The approach is identical to how we set the serial number. If you need further instructions, use the OOTB pattern Network Switch as a reference, review step 1–5.

NB: This way of setting serial numbers will work, but serial numbers differ somewhat from other CI attributes. In addition to being a CI attribute, a serial number should also be stored as a record in the designated serial numbers table in ServiceNow (cmdb_serial_number). You can view how this is done in OOTB pattern Network Switch, steps 5–7.

Learning Lab

Before sending you off on your SNMP journey, I will guide you through a more complex task.

Task: Implementing a mechanism to monitor the operability of fibre channel (FC) ports is essential when monitoring the health of network storage. Discover a storage switch’s FC ports using SNMP.

Steps to achieve objective:

• locate the OID pointing to the FC ports
• query for all ‘must-haves’ data points describing these ports, organize them in a temporary table
• ensure the discovered ports are included in payload
• set relationships and references between FC ports and Storage Switch

Locate OID

Open Brocade’s SNMP documentation: https://docs.broadcom.com/doc/FOS-82x-MIB-RM
Brocade® Fabric OS® MIB Reference Manual, 8.2.x

Skipping the usual SNMP detective work, navigate straight to page 64 section 4.6 Fibre Channel Port Group. The table lists all objects / OIDs in subtree 1.3.6.1.4.1.1588.2.1.1.1.6, along with a short description. Have a quick read, and try identifying which object / OID we need to query to retrieve the switch’s FC ports.

You guessed correctly! swFcPortTable points to a table containing information on each FC port. Note that for the pattern designer to traverse through each table entry, we have to move one step “down” the tree. The object for each port entry is swFCPortEntry, note down its OID (1.3.6.1.4.1.1588.2.1.1.1.6.2.1).

These objects are defined in SW-MIB. If you didn’t upload the required MIBs in the previous section, you need to do it now.

Configure SNMP Query

Insert a third step. Name it Get fcPortTable, select operation SNMP Query.

The next step is deciding on which values to query for. All OIDs listed in section 4.6 of Brocade’s SNMP documentation fall within the subtree 1.3.6.1.4.1.1588.2.1.1.1.6.2.1. Each OID within this subtree corresponds to a value describing a specific aspect or quality of the device’s FC port.

In other words, there are a whole lot of data points solely for describing the FC ports. With such an overload of data, let's identify the absolute ‘must haves’, and move from there.

The minimum information required on each port is the attribute functioning as the CI identifier. In CI Class Manager, look up ‘fibre channel port’, and locate the class’ identifier.

WWPN is the only identifier for the CI class

The port’s WWPN (World Wide Port Name) works as the identifier for all CIs of type FC port. Being the class’ sole identifier, assigning it a value is required to create a CI of this type. Try locating the OID pointing to a port’s WWPN (see page 67 of Brocade doc).

Which values to include besides the WWPN largely depends on the scope of your project. We will query for the following port qualities:

• WWPN ( CI Identifier ): .34
• Port Name: .36
• Operational Status: .4
• Port Type: .39

(To reduce redundancy, the list above omits the OID of the subtree, look at the digits after last dot (shortened to .34))

Step 3: ‘Get fcPortTable’ fully configured

The network call above queries the targeted Storage Switch for a list of all its FC ports. Each row points to an OID, telling the switch which attributes to include in its response.

Enter debug mode, and ensure that step 3 executes. In the list of temporary variables, locate fcPortTable, and validate the column names match the values specified in the SNMP query.

(Table contains dummy data)

Include FC ports in payload

As mentioned prior, the distinction between which variables to exclude and which to include in the payload, is drawn based on certain naming criteria.

All objects contained in the final payload must be linked to a valid CI class, this ensures it is properly processed when sent to SN identification engine. Without such a link, the identification engine is unable to classify an object as a member of the intended class. During payload construction, CI data is parsed into JSON objects, each assigned the JSON key className.

The value assigned to this key is derived from the name of the temporary variable that contained the data.

{
      "values": {
        "operational_status": "1",
        "wwnn": "10:00:00:00:00:00:00:20",
        "os_version": "v8.2.3a1",
        "name": "san_switch_a",
        "serial_number": "BRW20000000",
        "location": "SomePlace",
        "model_number": "Brocade6510",
        "model_id": "40-1000009-14",
        "sys_class_name": "cmdb_ci_storage_switch",
        "manufacturer": "Brocade"
      },
      "className": "cmdb_ci_storage_switch" 
  
}

Per now, data on fibre channel ports is stored in fcPortTable, with no way to determine which CI class the entries belong to. Copy the content of fcPortTable’s to a new table variable, set the name to be exactly ‘cmdb_ci_fc_port’, and voila, it will be included in the payload. When received by the designated SN instance, each JSON object is mapped to the intended CI class by reading this value.

Let's make all this happen. Insert a fourth step, and select the Transform Map operation. Add a precondition that checks for values in fcPortTable, allowing the step to be skipped if the table is empty.

Pre Conditions are usually a good idea

You may wonder why we didn’t set the FC ports directly to a table with the name ‘cmdb_ci_fc_port’. It's simply a personal preference of mine, I like to do all operations to the data in a buffer (here fcPortTable), and then copy to the final table.

Set Relationship and Reference

In Service Now’s CI Class Manager, look up ‘fibre channel port’. Under the tab Identification Rule, review the information listed.

The class is dependent; to register a CI of this type, we need to fulfill all dependent relationship rules. Select Dependent Relationships to see all dependent relationship rules applied to the class. Click on a vertex to review the specific relationship details.

The relationship must be configured to comply with the dependent relationship rule. To achieve this, a developer needs to identify the rule’s requirements for:

• Relationship type
• CI class of the child
• CI class of the parent

Review the above screenshot, and note the relationship type and child/parent class.

Type: Owned by (Owns:Owned by)
Child class: Fibre Channel Port
Parent class: Computer

A CI belonging to the class Computer, or any class that extends Computer may serve as the parent. The Storage switch class extends Computer, making it a valid parent.

Return to pattern designer. Insert a fifth step and choose the operation Create Relation/Reference.

The storage switch discovered is the parent, it ‘Owns’ all ports returned from the previous SNMP query, and vice versa all ports are ‘Owned by’ the storage switch. A developer may also define criteria for creating a relationship. We want ALL port entries to have a relationship linking it to the storage switch, no need for any creation criteria.

In addition to establishing a relationship, each port should include a reference to the switch. Store this reference in the ‘Computer’ column of the respective port

The Computer field references another CI, we use it to reference the switch containing the port

The reference direction option lets us choose how information is displayed:

• Parent to child: display parent CI on the child CI form. This will display the storage switch on the FC port form.
• Child to parent: display child CI on the parent CI form. Results in the Storage Switch form display a list of FC ports owned by the switch.

I would prefer having the list of FC ports owned to display on the switch’s form. Reference direction is Child to parent.

The payload will now contain an object describing the main CI (storage switch), an array of objects describing FC ports, and an array of objects with data on the relationships between the switch and ports.

Closure Notes

Congrats on completing the lab! It may take some time for all this to sit. Keep practicing, learning pattern designer makes it possible to add unique functionality to your organization’s Configuration Management practices.

For your next move, I strongly recommend having a look at the module Discovery → Pre / Post Processing. Learn how to intercept the processing of payloads through the power of JavaScript. Such scripting capabilities, combined with knowledge of pattern designer, give you complete control over your data, opening a whole new world of possibilities! 🦸‍♀️

If you want an article on Pre/Post processing scripts, leave a comment 🥰