Profile Activation and Configuration
  • Dark
    Light
  • PDF

Profile Activation and Configuration

  • Dark
    Light
  • PDF

Profile Activation

To set an active IAP profile:

  1. Verify installation of RabbitMQ plugins. Please see the section on RabbitMQ Installation in the IAP Admin Guide.
  2. Configure IAP profile.
  3. Configure service config of each event producer which requires event deduplication.

Profile Configuration

In order for IAP to use the installed RabbitMQ plugins, the event deduplication feature must be enabled in the rabbitmq section of the active profile. See below for an example configuration setting that can be added to the IAP Profile:

Example

The following is a sample profile configuration with exhaustive defaults.

{
  ...,
  "rabbitmq": {
    ...,
    "adminPort": 15672,
    "eventDeduplication": {
      "active" : false,
      "cacheSize" : 1000,
      "cacheTtl" : 100,
      "cachePersistence" : "memory",
      "messagePreservation" : {
        "strategy" : "automatic",
        "tempQueueName" : "iap_event__TMP__",
        "shovelName" : "preserve_iap_events",
        "shovelAckMode" : "on-confirm",
        "shovelAddHeaders" : false,
        "shovelPrefetchCount" : 1000,
        "shovelReconnectDelay" : 5000,
        "maxTransferWait" : 5000,
        "transferWaitPollIncrement" : 100
      }
    }
  }
}

adminPort

The adminPort property specifies the port the RabbitMQ Management API is exposed on. This API is used to manage event deduplication configuration and must be exposed and specified in your profile settings in order for event deduplication to operate correctly.

eventDeduplication

Use this table to set eventDeduplication properties for the active profile.

Property Type Default Description
active Boolean False Turns event deduplication functionality on or off at the system level.
cacheSize Integer 1000 Denotes the maximum number of messages that will be cached when deduplicating events.
cacheTtl Integer 100 Denotes the time in milliseconds the unique identifier for a message will be cached after it is published.
cachePersistence Enum
("memory",
"disk")
Memory Denotes which type of memory to use when caching message identifiers.
messagePreservation Object Enabled Container for settings related to preservation of transient messages left from a previously running IAP cluster. See section below.

messagePreservation

It is theoretically possible to have unprocessed messages left in the event queue if a catastrophic error occurs (lost network connectivity to RabbitMQ or an event is published just as IAP restarts). For this reason, IAP offers configurable options for preserving unprocessed event messages. The default options for this group of settings are enabled by default for safety.

Property Type Default Description
strategy Enum
("automatic",
"manual",
"off")
Automatic Strategy for handling transient messages. Enum options are:
automatic will move residual messages to a temporary queue while the event queue is recreated, and then restore them afterwards.
manual will abort IAP startup when messages exist in the event queue so that an engineer can rectify the situation.
off will update event deduplication settings regardless of transient messages, destroying them in the process. The off option is not recommended, but may be necessary in some situations at the discretion of the administrator.
tempQueueName String iap_event__TMP__ Name of temporary container queue. Used to hold transient messages while recreating the iap_event queue with new settings.
shovelName String preserve_iap_events Name of the temporary queue used to move messages to and from the temporary container queue.
shovelAckMode Enum
("on-confirm",
"on-publish",
"no-ack")
on-confirm Refer to the official shovel plugin documentation for detailed information on the ack-mode setting.
shovelAddHeaders Boolean False Denotes whether or not the x-shovelled header should be set on messages processed by the shovel.
shovelPrefetchCount Integer 1000 Number of messages the shovel will transfer at a given time.
shovelReconnectDelay Integer 5000 Duration in milliseconds to wait if the broker becomes disconnected.
maxTransferWait Integer 5000 Duration in milliseconds to wait for message transfer both to and from the temporary queue.
transferWaitPollIncrement Integer 100 Increment in milliseconds used when polling the message count in the queue being drained by the transfer shovel.