Event Service Configuration

Event Listening

The event deduplication feature is designed to handle situations where the same event might occur from multiple sources, especially in the context of highly-available architecture. If multiple downstream resources can report the same event, or if multiple IAP instances are listening to events from the same downstream resource, multiple event messages may be broadcast in the IAP event system for the same downstream occurrence. In this case, actions listening for those events will execute multiple times. In most cases, however, the desired behavior is to respond only once for each distinct occurrence.

For example, event deduplication may be required if an instance of NSO is observed by a cluster of IAP servers. In this scenario, each IAP server will maintain its own individual connection to the NSO server. NETCONF events produced by the NSO server will be observed by all running instances of IAP, resulting in multiple messages being generated in the IAP event system for a single NETCONF event. In this case, IAP event deduplication may be configured to ensure that only one message is emitted to the event bus so that there is only one response to the event from the system as a whole.

Configuring Event Deduplication

In the active IAP profile, you will need to set the service config of each event producer which requires event deduplication. Once the active profile is configured to enable event deduplication, each service for which events are to be deduplicated must then be configured.

The most important settings shown below are active and uniqueProps, which determine the on/off status of the feature, as well as the set of message properties that will be used to determine which messages are duplicates. The uniqueProps allow you to handpick the fields that will be used to evaluate if two events are to be regarded as duplicates. For example, if the values of the three fields specified below ("/value/device", "/value/event", "/value/timestamp") are the same between two events, the event is considered as a duplicate. If any of these are different, then the events are not deduplicated.

Below is a sample service config for eventDeduplication settings.

{
  ...,
  "properties" : { ... },
  "eventDeduplication": {
    "active" : true,
    "cacheTtl" : 100,
    "algorithm" : "md5",
    "uniqueProps" : [
      "/value/device",
      "/value/event",
      "/value/timestamp"
    ]
  }
}

eventDeduplication

Use this reference table to help set eventDeduplication properties for the service config.

algorithm

The algorithm setting is unlikely to be required, but in the case that it is, available options are discoverable via openssl list -digest-algorithms, or openssl list-message-digest-algorithms on older systems. The library used to create a message digest uses the system's openssl utility to produce a result.

uniqueProps

The uniqueProps setting is used to tell the deduplication system which fields define a unique message. The fields specified in this property are used to extract a subset of the message which is then hashed to provide a unique value to the deduplication system.

For example, if the uniqueProps property specifies the fields device, event and timestamp, the following messages will be considered as redundant to each other, and two of them will be dropped while the first is emitted through the event system.

{
  "device": "ios1",
  "event": "CRASH",
  "timestamp": 1592329817,
  "remediation": "REBOOT"
}

{
  "device": "ios1",
  "event": "CRASH",
  "timestamp": 1592329817,
}

{
  "device": "ios1",
  "event": "CRASH",
  "timestamp": 1592329817,
  "region": "us-east-1"
}

When using inclusive configuration (shown below as JSON Pointer Array and Inclusive Mask Object), it is mandatory that uniqueProps be specified explicitly. In the event that none of the specified fields are found in a message, it is considered unique and subsequent messages like it will not be deduplicated.

JSON Pointer Array

When uniqueProps is an array, it must contain a list of JSON pointer expressions. These expressions tell IAP which properties should be used when comparing messages, forming an inclusive list.

[
  "/value/device",
  "/value/event",
  "/value/timestamp"
]

Inclusive Mask Object

The uniqueProps setting may also be specified as an object with true or false as its leaf values (example below). In this form, uniqueProps acts as a 'mask' object. Keep in mind, true and false leaf values may not be mixed, as this causes ambiguity in the mask object in some situations.

The example below is equivalent to the JSON Pointer Array example above.

{
  "value": {
    "device": true,
    "event": true,
    "timestamp": true
  }
}

The inverse of the above is also possible for use in case the messages from an event source are deduplicated based on all but a known set of properties. For example, if the message in question also contains properties for region and severity, the following mask object would be equivalent to both of the above examples:

{
  "value": {
    "region": false,
    "severity": false
  }
}

Email Event Triggers

Under certain conditions, the email adapter is known to create duplicate jobs when being used as an event trigger in Operations Manager. This issue can occur when emails are processed at a high rate and can be remediated by specifying a messageId field in the uniqueProps setting of the service configuration as shown in the example below.

{
  ...,
  "properties" : { ... },
  "eventDeduplication": {
    "active" : true,
    "cacheTtl" : 10000,
    "algorithm" : "md5",
    "uniqueProps" : [
      "/messageId"
    ]
  }
}