Troubleshoot

Diagnose and resolve common logging problems.

Before you begin

• Administrator access to Itential Platform
• Platform version (check Admin Essentials > About)
• Understanding of log severity levels

Collect logs for support

Choose the appropriate collection method based on your needs and whether you require JSON formatting.

Method comparison

Use journalctl

Collect logs using journalctl to capture all platform output in a single file.

Identify your service name:

Check running service:

systemctl status itential-platform

Collect all logs:

journalctl -u itential-platform > itential-logs-$(date +%Y%m%d).log

Collect from time range:

# Last hour
journalctl -u itential-platform --since "1 hour ago" > logs.txt

# Specific range
journalctl -u itential-platform --since "2024-11-20 10:00:00" --until "2024-11-20 15:00:00" > logs.txt

Collect log files directly

Collect log files directly from the file system to preserve JSON formatting.

# Platform 6
tar -czf itential-logs-$(date +%Y%m%d).tar.gz /var/log/itential/

# Platform 2023.2 and earlier
tar -czf pronghorn-logs-$(date +%Y%m%d).tar.gz /var/log/pronghorn/

Before sharing logs with third parties: Review logs for sensitive data from third-party systems. Itential Platform excludes AAA sensitive data but does not modify third-party payloads.

Missing log messages

If expected log messages don't appear in log files, verify your logging configuration and directory permissions.

Check log level

Each log level excludes lower-severity levels. Setting log_level to info excludes debug messages.

Platform 6:

1. Navigate to Admin Essentials > Configuration > Logging
2. Check log_level setting

Platform 2023.2 and earlier:

1. Navigate to Admin Essentials > Profiles > [your profile]
2. Click Configure tab
3. Select loggerProps
4. Check log_level setting

Enable verbose logging temporarily

Temporarily increase log verbosity to capture detailed diagnostic information.

1. Set log_level to debug
2. Save and restart Platform
3. Reproduce the issue
4. Return log_level to info or warn

Enable debug for specific adapters

Enable debug logging for individual adapters to minimize impact on overall system performance.

1. Navigate to Admin Essentials > Adapters
2. Select adapter
3. Click Logging tab
4. Set File Log level to debug
5. Save and restart adapter

Check component overrides

Verify that individual components haven't overridden global logging settings.

1. Navigate to Admin Essentials > Applications, Adapters, or Integrations
2. Select component
3. Click Logging tab
4. Verify log level
5. Reset to global settings if needed
6. Restart adapters (applications apply immediately)

Verify log directory

Ensure the log directory exists with proper permissions for the Platform process to write logs.

Check directory exists with proper permissions:

# Platform 6
ls -ld /var/log/itential

# Platform 2023.2 and earlier
ls -ld /var/log/pronghorn

Directory should be owned by itential with 755 permissions.

Create directory if missing:

# Platform 6
sudo mkdir -p /var/log/itential
sudo chown itential:itential /var/log/itential
sudo chmod 755 /var/log/itential

# Platform 2023.2 and earlier
sudo mkdir -p /var/log/pronghorn
sudo chown itential:itential /var/log/pronghorn
sudo chmod 755 /var/log/pronghorn

Restart Platform after creating directory.

Check system journal

Check the system journal for logging initialization errors or permission issues.

journalctl -u itential-platform --since "10 minutes ago"

Look for permission problems, configuration errors, or disk space constraints.

Logs missing from expected time period

If logs from a specific time period are missing, log rotation may have deleted them before you could retrieve them.

Check rotation settings

Verify your current rotation configuration meets your retention requirements.

Platform 6:

1. Navigate to Admin Essentials > Configuration > Logging
2. Check log_max_file_size and log_max_files values

Platform 2023.2 and earlier:

1. Navigate to Admin Essentials > Profiles > [your profile]
2. Click Configure tab
3. Select loggerProps
4. Check log_max_file_size and log_max_files values

Calculate retention period

Determine how many days of logs your current settings retain:

Days of retention = (log_max_file_size × log_max_files) / estimated daily log volume

Example calculation:

• log_max_file_size: 10485760 (10 MB)
• log_max_files: 10
• Estimated daily log volume: 50 MB

Retention = (10 MB × 10) / 50 MB per day = 2 days

Adjust rotation settings

Increase retention by adjusting log_max_file_size or log_max_files before data loss occurs.

Production recommendations:

• log_max_file_size: 52428800 (50 MB)
• log_max_files: 50 to 100
• This provides approximately 7-30 days of retention depending on log volume

Alternative: Reduce log verbosity instead of increasing file limits:

1. Set log_level to info or warn
2. Disable debug logging on individual components
3. This extends retention without increasing disk usage

Update settings, save, and restart Platform.

Excessive log volume

If logs rotate too quickly or consume excessive disk space, reduce verbosity and adjust rotation settings.

Reduce verbosity

Production systems should use info or warn log levels. Never use trace or spam in production.

Platform 6: Admin Essentials > Configuration > Logging

Platform 2023.2 and earlier: Admin Essentials > Profiles > [your profile] > Configure > loggerProps

Set:

• log_level: info or warn
• console_level: warn or error

Save and restart Platform.

Note: Console logging has higher performance impact than file logging.

Adjust rotation settings

Configure rotation settings to balance disk usage with retention requirements.

Total disk usage = log_max_file_size × log_max_files

Production recommendation:

• log_max_file_size: 52428800 (50 MB)
• log_max_files: 50 to 100

Update in logging configuration, save, and restart.

Enable structured logging

Structured logging reduces log volume compared to standard format logs.

Platform 2023.2 or 6.2+ supports structured logging, which reduces log volume. See Configure logging.

Disk space full

If log files fill available disk space, archive or delete old logs and adjust rotation settings.

Check disk usage:

df -h /var/log/itential

Archive old logs:

tar -czf itential-logs-$(date +%Y%m%d).tar.gz /var/log/itential/itential-platform.log.*
mv itential-logs-*.tar.gz /backup/location/

Delete rotated logs older than .5:

rm /var/log/itential/itential-platform.log.[5-9]*
rm /var/log/itential/itential-platform.log.[1-9][0-9]*

Restart Platform after freeing space.

Reduce log_max_files in logging configuration to prevent recurrence.

Performance issues

If Platform performance decreases after changing logging settings, reduce log verbosity and check system resources.

1. Check if log_level or console_level is set to debug or more verbose
2. Reset to info/warn (file) and warn/error (console)
3. Check individual components for verbose logging
4. Monitor system resources:

   top
   iostat -x 1

Parse log errors

If you encounter errors when parsing log files, verify the log format and JSON structure.

Mixed log formats

During structured logging migration, logs may contain both formats. Configure your parser to:

1. Check for message field (structured)
2. Fall back to @message field (standard)
3. Handle context.legacy_args arrays

Preserve JSON formatting

journalctl doesn't preserve JSON structure. Collect logs directly to maintain formatting:

cat /var/log/itential/itential-platform.log

Validate JSON

Test JSON validity to ensure logs are properly formatted:

cat /var/log/itential/itential-platform.log | jq '.' > /dev/null

Contact support with examples and your Platform version if errors appear.

Next steps

• Configure logging
• View and search logs
• Logging best practices for developers
• Contact support