Throttle Properties

Updated on 10 Jul 2023

Dark

Light
PDF

Article summary

Did you find this summary helpful?

Thank you for your feedback

Definition

All throttle properties are contained within the throttle object in the IAP Service Instance Configuration for the adapter. These tell the adapter libraries how to handle throttling of requests to the other system. Throttling is not a common capability that is used, but if you find that the other system either can not handle heavy loads, or has concurrent license limitations, it may require the use of throttling. This is where the adapter will queue up requests and only send them to the other system as that properties tell it to. Additionally, throttling should only be used when traffic/usage has peaks and valleys. Valleys will allow the adapter time to work through and reduce the queue back to 0. Without time to do that, eventually, requests to the adapter will be rejected because the queue is full.

Property	Type	Required	Description
`throttle_enabled`	boolean	Yes	Turns throttling on for the adapter.
`number_pronghorns`	integer	Yes (if enabled)	Number of Itential platforms over which the throttling queue needs to work. Setting it >1 is currently non-functional and the feature is undergoing a fix.
`sync_async`	string	No -> sync	Not currently used; reserved for future use.
`max_in_queue`	integer	Yes	The maximum number of requests the adapter should hold in the throttling queue. Requests over the maximum will be rejected immediately to prevent the queue from growing without bound. Note: The Itential platform can hold thousands of items in the queue, but how fast they get responses will depend on the other system.
`concurrent_max`	integer	Yes	The maximum number of requests that can be outstanding with the external system at any given point in time. The adapter attempts to always have this number of requests active while working down the queue.
`expire_timeout`	integer	Yes	Defines the time in milliseconds (ms) after each request is sent that the adapter needs to wait before sending the next request. This allows time for a request slot or license to become available again in the other system.
`avg_runtime`	integer	Yes	Defines the time in milliseconds (ms) that the adaper should expect the external system to respond. This number is used as a seed time for determining queue waits. It is replaced by average run time of the last 25 requests as the adapter processes those requests.
`priorities`	array	No	Defines priority rules for handling requests within the throttle queue. This allows you to self-define a small or large number of rules for prioritization. `value` (integer): Identifies a prioritization rule configuration. This value will also be provided in the request object priority so that the adapter knows which prioritization rule to apply to the request `percent` (integer): Defines the percent of the queue that comes before this priority. Example: 0% will result in the request being put first in the queue, and 25% means the request will be put 25% of the way through the queue. Thus, if there are 100 requests in the queue, the new request will be placed at 25.

Example Scenario

The system can only handle five (5) concurrent requests.
There is only one IAP running with this adapter.
Whenever the queue has 500 requests, all other requests should be rejected to indicate the queue is full so that the adapter has a chance to catch up with the other requests.
The other system does not require any license recovery time.
Each request to the system should take approximately 400ms.
- Approximating this is acceptable as it is only used as an initial seed.
- The adapter utilizes actual call averages while the calls are in progress.
Three different priority queues should be created for requests.
- The 1st is for blocking issues that should be at the front of the queue.
- The 2nd is for requests which are still critical but not critical enough to be at the front of the line. They are put in the queue at 10% – in front of 90% of the current request.
- The 3rd is for requests that are not critical but should be prioritized above the normal usage - these are set at 50%.
- If there is no priority with the request it will be at the end of the throttle queue - these are set at 100%.

IAP Service Instance Configuration Properties for the Adapter

"throttle": {
 "throttle_enabled": true,
 "number_pronghorns": 1,
 "sync_async": "sync",
 "max_in_queue": 500,
 "concurrent_max": 5,
 "expire_timeout": 0,
 "avg_runtime": 400,
 "priorities": [
   {
     "value": 1,
     "percent": 0
   },
   {
     "value": 2,
     "percent": 10
   },
   {
     "value": 3,
     "percent": 50
   }
 ]
},

Was this article helpful?

What's Next

Mongo Properties

Table of contents

Definition