Configure rate limiting

This topic explains how to configure rate limiting for the Workflow Engine's task workers.

Prerequisites

• Platform 6.3 or newer installed
• System administrator access to modify Platform configuration
• Access to Workflow Engine configuration API (for runtime configuration)

Configuration parameters

Rate limiting parameters are set at the Platform instance level in environment variables, your .properties configuration file, or your profile document.

Configuration parameters explained

Task worker rate limit

Primary rate limit control for task execution. This limit applies across all task workers in the Platform instance.

• Set to positive integer to enable rate limiting
• Set to 0 to disable completely
• Represents the maximum number of tasks that can be executed per period

With multiple task workers:
If you configure 3 task workers with a rate limit of 100 tasks per period, the total throughput is 100 tasks per period (~33 tasks per period per worker).

Task worker rate limit period

Defines the time window (in seconds) for the rate limit.

• Set to positive integer (seconds)
• Works with task_worker_rate_limit to define execution rate
• Example: task_worker_rate_limit=100 with task_worker_rate_limit_period=1 = 100 tasks per second

In most cases, use the default period of 1 second unless you need to rate limit to slower than 1 task per second.

Task worker thread count

Controls how many task worker threads run within the Platform instance.

• Each worker processes tasks concurrently
• All workers share the configured rate limit
• More workers can improve CPU utilization and throughput
• Example: Three task workers at 100 tasks per period total (33 per worker) can better utilize available CPU while maintaining the same rate limit
• Default: 1 task worker

Configuration approach

Rate limiting configuration can be set at startup and can be adjusted at runtime through the Workflow Engine API.

At startup:

• Set in Platform-level environment variables, your .properties configuration file, or your profile document
• Configuration loads when Workflow Engine starts
• Persists across restarts

At runtime:

• Adjust via Workflow Engine HTTP API without restart
• Changes take effect immediately
• Not persisted across restarts unless also updated in configuration files

Configure at startup (persistent)

Step 1: Determine appropriate rate limits

Choose rate limits based on:

• System resource capacity (CPU, memory)
• Typical workflow volumes and patterns
• Peak usage times and volumes
• Downstream service capacity

Monitor your system under typical load and adjust based on observed performance.

Step 2: Set configuration

Set rate limit parameters in your Platform-level configuration:

task_worker_rate_limit=100
task_worker_rate_limit_period=1
task_worker_thread_count=2

Step 3: Restart the Platform

Restart your Itential Platform instance to load configuration changes.

For more information, see Restart.

Step 4: Verify configuration

Verify the configuration has been applied:

Option 1: Check system logs

Check system logs for rate limit initialization messages:

1. Open Workflow Engine log files
2. Look for entries confirming rate limit settings:
   • Task rate limit enabled at X tasks per Y seconds
   • Number of task workers: Z
3. Verify logged values match your configuration

Option 2: View in Admin Essentials

View the current configuration in Admin Essentials:

1. Navigate to Admin Essentials
2. Select Configuration from the left-side navigation
3. Locate the task_worker_* configuration parameters
4. Verify values match your configuration

If values don't match or no initialization messages appear:

• Check configuration file syntax
• Verify configuration is at Platform level
• Ensure Platform version is 6.x or newer
• Review system logs for configuration errors

Configure at runtime (immediate, not persistent)

Use the runtime API to adjust rate limits immediately without restarting services. Changes take effect right away but are not persisted across restarts.

Step 1: Call the configuration API

Parameters:

• task_worker_rate_limit
• task_worker_rate_limit_period
• task_worker_thread_count

Example API call:

curl -L -g -X PUT 'http://localhost:3000/workflow_engine/workers/rate_limit?token=••••••' \
-H 'Content-Type: application/json' \
-d '{
    "rateLimit": 10,
    "rateLimitPeriod": 1
}'

Step 2: Verify configuration

Query the configuration API to confirm new values:

curl -L -g -X GET 'http://localhost:3000/workflow_engine/workers/rate_limit?token=••••••'

Step 3: Update configuration files for persistence (recommended)

For production deployments, update configuration files to match your runtime settings to ensure they persist across restarts.

Configuration examples

Basic task rate limiting

Enable task rate limiting at 50 tasks per second:

task_worker_rate_limit=50
task_worker_rate_limit_period=1

Uses default: 1 task worker thread

Task rate limiting with multiple workers

Enable task rate limiting with multiple workers to better utilize CPU:

task_worker_rate_limit=150
task_worker_rate_limit_period=1
task_worker_thread_count=3

Result: 150 tasks per second total, ~50 tasks per second per worker

Adjust for CPU cores

Small instance (4 cores):

task_worker_rate_limit=100
task_worker_rate_limit_period=1
task_worker_thread_count=2

Result: 100 tasks per second total, ~50 tasks per second per worker

Large instance (12 cores):

task_worker_rate_limit=200
task_worker_rate_limit_period=1
task_worker_thread_count=3

Result: 200 tasks per second total, ~67 tasks per second per worker

Rate limiting slower than 1 task per second

Configure rate limiting for very slow task execution:

task_worker_rate_limit=1
task_worker_rate_limit_period=5

Result: 1 task per 5 seconds (0.2 tasks per second)

Disable rate limiting

task_worker_rate_limit=0

Task workers process tasks as quickly as possible without rate limiting.