Tune rate limits

This topic explains how to iteratively adjust rate limits to optimize performance.

For initial configuration, see Configure rate limiting.

Iterative tuning process

Start with conservative rate limit values based on your system capacity.

Monitor system resources under typical load: CPU utilization, memory usage, and disk I/O.

Gradually increase the rate limit if resources allow.

Test with realistic workflow volumes.

Adjust based on observed patterns.

Signs to increase rate limit

Tasks consistently queuing during normal operation
Queue depth steadily increasing
System resources underutilized (less than 70% CPU)
Task workers have idle capacity

Signs to decrease rate limit

Sustained high CPU utilization (greater than 80%)
Memory pressure or out-of-memory errors
Frequent task failures
Downstream services reporting overload
System becoming unresponsive

When optimal configuration is achieved

Workflows complete in acceptable timeframes
System resources well-utilized (60–80% CPU)
Queue depths stable or decreasing
Minimal rate limiting impact during normal operation
System responsive during peak loads

When to adjust specific parameters

Increase task rate limit

Resources underutilized and workflows slow
Queue depths consistently high
Business requirements not being met

Decrease task rate limit

System showing signs of overload
High resource utilization
Downstream services struggling

Adjust rate limit period

Adjust the period only if you need to rate limit to slower than one task per second:

task_worker_rate_limit=1 with task_worker_rate_limit_period=2 = one task per two seconds
task_worker_rate_limit=1 with task_worker_rate_limit_period=10 = one task per 10 seconds

For rates of one task per second or faster, keep the default period of one second.