Throttling

Adapters can throttle outbound traffic to a downstream system. When throttling is enabled, the adapter accepts a large volume of incoming requests — potentially thousands in a short period — queues them internally, and forwards them to the downstream system at a controlled, lower rate of concurrency. This continues until the queue is cleared.

Throttling is appropriate when the downstream system cannot handle high concurrency, or when it has a licensing constraint that limits the number of simultaneous requests it can process.

Considerations before enabling throttling

Throttling has significant side effects. Enable it only after reviewing the following:

Set a queue limit. Configure a maximum queue size in the throttle properties. Once the limit is reached, the adapter rejects new requests. Without a limit, the queue can grow unboundedly.

Expect slower responses. Requests wait in the queue until the adapter determines the downstream system can accept them. Response times can be many minutes.

Traffic must have peaks and valleys. Throttling works best when periods of high traffic are followed by quieter periods during which the adapter can drain the queue. If requests arrive at a sustained high rate with no valleys, the queue will grow continuously, resulting in delayed responses and rejected calls.

Use a long token timeout. If each call requires a token, token fetching consumes time that could otherwise be used to drain the queue. Set token_timeout to 600000 ms or higher when using throttling.

Avoid intermittent health checks, or use a long interval. Health checks are added to the throttle queue alongside regular requests, which can cause the queue to fill with health check traffic. Either disable intermittent health checks when throttling is active, or set the interval to 900000 ms or greater.

Do not mix throttling at multiple stages. Some downstream systems already throttle inbound calls at their own layer. Adding adapter-level throttling on top of this can cause unforeseen behavior. For example, the Itential Automation Gateway already throttles inbound requests, so adapter-level throttling is not recommended when integrating with adapter-automation_gateway.