Python virtual environments

When IAG executes a python-script service, ansible-playbook service, or a runCode task, it builds an isolated virtual environment on the runner containing the dependencies that execution requires. This topic explains how IAG manages those environments and how to configure automatic cleanup.

How IAG manages virtual environments

IAG caches virtual environments so that dependencies are installed once and reused across executions. Each environment is keyed to a specific set of dependencies — when the requirements for a service or task are unchanged, IAG reuses the existing environment rather than reinstalling. On each execution, IAG:

1. Checks whether a virtual environment already exists for the required dependency set.
2. If a matching environment exists, reuses it.
3. If no matching environment exists, builds a new one by installing the required dependencies, then caches it for future use. Because environments are cached and reused, repeated executions with the same dependency set incur no reinstall overhead.

For python-script and ansible-playbook services, dependencies are read from a requirements.txt file in the service’s working directory. python-script services also support pyproject.toml as an alternative to requirements.txt.

For runCode tasks, dependencies are specified at design-time in the task’s Code Editor dialog.

Virtual environment pruning

Without cleanup, cached virtual environments accumulate on disk over time. As of IAG 5.4, IAG includes a background pruner that automatically removes environments that have been idle beyond a configurable retention period.

Two configuration parameters control pruner behavior:

Both settings are configurable in gateway.conf under the [application] section or via environment variables. For more information, see Application variables.

Tune the retention period

If you set venv_retention_period too low, IAG may prune an environment between uses and force a full dependency reinstall on the next execution, adding latency to that run.

When tuning the retention period, consider how frequently your services and tasks run. If some run infrequently — for example, monthly maintenance scripts — set the retention period to exceed the longest interval between executions for those services.