Prometheus Metrics & Configuration

Data Metrics

Itential Automation Platform exposes various system metrics for scraping via the /prometheus_metrics route. A prometheus server can be run and configured to scrape /prometheus_metrics which in turn will visualize that data in various charts and graphs.

Prometheus can be used for collecting time series numeric data, which is useful for testing and diagnostic purposes, such as during an IAP outage. Metrics are returned in a text exposition format that is understood by prometheus servers of any version:

# HELP iap_unique_sessions Number of unique IAP sessions
# TYPE iap_unique_sessions gauge
iap_unique_sessions 1

# HELP iap_active_sessions Number of active IAP sessions
# TYPE iap_active_sessions gauge
iap_active_sessions 1

# HELP iap_total_api_calls Total count of api calls since IAP start
# TYPE iap_total_api_calls gauge
iap_total_api_calls 1

# HELP iap_active_jobs Number of active jobs
# TYPE iap_active_jobs gauge
iap_active_jobs 17882

# HELP iap_context_manager_map_size number of entries in the shared context map
# TYPE iap_context_manager_map_size gauge
iap_context_manager_map_size 0

# HELP iap_context_manager_stale_message_count number of old entries in the context map
# TYPE iap_context_manager_stale_message_count gauge
iap_context_manager_stale_message_count 0

# HELP iap_memory_heap_usage_percentage percent of heap used
# TYPE iap_memory_heap_usage_percentage gauge
iap_memory_heap_usage_percentage 92.39992269241

# HELP iap_total_memory_heap_size total number of bytes allocated for this node process's heap
# TYPE iap_total_memory_heap_size gauge
iap_total_memory_heap_size 46194688

# HELP iap_memory_heap_used_size number bytes currently in use in this node process's heap
# TYPE iap_memory_heap_used_size gauge
iap_memory_heap_used_size 42683856

# HELP iap_cpu_user_usage user CPU time usage of IAP process
# TYPE iap_cpu_user_usage gauge
iap_cpu_user_usage 9479497

# HELP iap_cpu_system_usage system CPU time usage of IAP process
# TYPE iap_cpu_system_usage gauge
iap_cpu_system_usage 1884960

Configuration File

Prometheus servers are configured using a .yml configuration file. By default, this file is prometheus.yml. The metrics_path option must be changed to /prometheus_metrics for IAP since prometheus scrapes /metrics by default. For more information on configuration, see the prometheus.io docs site.

Below is an example prometheus config file.

# global config
global:
  # frequency to scrape targets
  scrape_interval:     2s
  evaluation_interval: 5s

# A scrape configuration containing exactly one endpoint to scrape:
scrape_configs:
  - job_name: 'pronghorn'

    static_configs:
    # hostname and port to scrape
    - targets: ['host:port']

    # scrape non-default mertrics path for IAP
    metrics_path: /prometheus_metrics

    basic_auth:
      username: 'admin@pronghorn'
      password: 'admin'

    # Protocol scheme for requests
    scheme: https

    tls_config:
      # client cert authentication
      cert_file: path/to/crt
      key_file: path/to/key

      # disable validation of server certificate
      insecure_skip_verify: false

Metrics Collected

The following IAP metrics are collected.