Throttling

Throttling

Adapters have the ability to throttle the flow of traffic to the downstream system. The adapter can handle taking in a large number (thousands) of requests in a short period of time, queue those requests up and then send them to the downstream system in a smaller number of concurrent requests. The adapter will continue to send requests to the downstream system over some period of time until all the requests in the queue have been processed.

Why: This is done when the downstream system either can not handle the large load of requests at the same time and so they have to be spread out over a larger time interval or it has some concurrent license criteria that will prevent it from handling more than n concurrent requests.

While the adapter handles this, this capability should only be used when careful consideration has been given to potential side effects.

1. You should set some limit to the number of items the adapter can queue (this is in the adapter throttle properties). After that, the adapter will start rejecting requests.
2. You should expect that responses will take longer because the request will sit in the queue until it is determined that the downstream system is capable of handling the request. This responses can be many minutes.
3. Adapter requests should come in peaks and valleys. During a peak level of traffic, the adapter will build up its queue. During a valley the adapter is not getting a lot of requests and so it is able to work down the queue. The ability to work down the queue is critical to successful throttling - if there is no time for the adapter to work down the queue the consequences will be a lot of delayed responses and rejected calls.
4. If each call has to get authentication information, that can further consume time taken for clearing the queue. We strongly recommend have a token timeout of 600000ms or higher when using throttling to get good use out of the token.
5. If you are running intermittent healthchecks, they will be added into the queue and get worked along with all other requests. This can result in the throttling queue filling up with healthchecks. We strongly recommend either not using intermittent healthchecks with throttling or using an interval of 900000ms or greater.
6. Mixing throttling at different stages can cause unforeseen consequences and is not recommended. As an example, the Itential Automation Gateway Server throttles calls into that system. With this in mind we do not recommend throttling be used to this system (adapter-automation_gateway)