Web Server Access Log

This guide explains how to use a Web Server Access Log to see a commonly used log file which shows the calls made to the web server within Itential Automation Platform (IAP).

Common Log Format

The log format for the Web Server Access Log is a stringified JSON object with each key mapping to a part of the Common Log Format. Since this is a JSON format, the ordering of the keys is not guaranteed. The keys, in the order they appear, in a traditional Common Log Format are referenced below.

Configuration

The Web Server Access Log currently has two different configurable properties that can be edited within an IAP profile by updating loggerProps in the Profile Properties. Both properties can be found in the webserver section of the Configure tab. Example input is shown in the figure below.

Figure 1: Profile Properties

Log Rotation

There is no automatic log rotation built into the Web Server Access Log which means that configuring an external log rotator is very important. One very useful tool to accomplish this is logrotate, a Linux tool that can automatically rotate a log file based on configured parameters. It can be installed simply by using your package manager and running:

RHEL/CentOS

yum install logrotate

Verify that logrotate is installed, and then check the configuration file, which for this example will be at /etc/logrotate.conf. There may be other ways to handle configuration such as inside the /etc/logrotate.d/ folder. The following is an example configuration where the webserver logfile is named webserver.log and is located at /var/log/pronghorn.

/var/log/pronghorn/webserver.log {
  rotate 10
  size 10M
  nocompress
}

In the above example, the logs will be rotated a maximum of 10 times before the old log files will be deleted. It will only rotate the logs when their size exceeds 10 megabytes, as specified by 10M in the configuration file. Finally, it will not compress files when it rotates them, as defined by nocompress, which it would otherwise do automaticallly if not specified.

Below is another possible configuration file:

/var/log/pronghorn/webserver.log {
  weekly
  rotate 20
  postrotate
      echo Hello World
  endscript
}

In this example, the logs will rotate weekly instead of rotating based on any fixed size. It will also rotate a maximum of 20 times. Finally, there is a postrotate script included, which means that in this case, any time the logs are rotated it will echo Hello World to the console. It may be useful to configure a custom script to run whenever logs rotate.

Running this command will start the log rotation:

    logrotate /etc/logrotate.conf

If you wish to force a rotation immediately, even when the various initiators of a rotation such as file size have not been met, run the following command:

    logrotate -f /etc/logrotate.conf

Querying Log Files

The log files can be queried using basic command line tools such as grep to search the file for a given pattern.

Using GREP or Similar Command Line Tools

If a user wanted to find any API requests made by a user named admin for the log file named webserver.log, they could use the following command:

grep '"remote_user":"admin"' webserver.log

In order to retrieve all the times a user has made a request to a specific route, such as GET /health/system for example, they could use the following command:

grep -E '"url":"/health/system"' webserver.log | grep '"verb":"GET"'

There are many other ways grep and similar command line tools can be used to query information from the JSON logs. Since all data values are preceded by a key, such as url or remote_user, it is possible to do standard queries on any value.

Using Third-Party Visualization Tools

The log format used by the access log should work with various third-party tools which support a standard JSON log format. These can be used for more advanced queries or visualization of logging information.