Full Installation Method

Full Installation

The following step-by-step guide is meant to explain each phase so that the user can deviate from the Quick Installation Method where appropriate for their environment. It assumes you have root access to the system, or that you will utilize sudo on all commands up to Continue As Non Root User.

Enable Repositories

Automation Gateway requires the following yum repositories.

• epel-release
• ius-release

yum -y install epel-release
yum -y install \
https://repo.ius.io/ius-release-el7.rpm \
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Update System Packages

To ensure the latest packages are installed, perform a yum update.

yum -y update

Install System Dependencies

Note: Please refer to Release Notes for a more exhaustive list of requirements and dependencies.

As Automation Gateway is a Python application, it has multiple system dependencies that need to be installed.

yum -y install gcc-c++ make yaml-cpp yaml-cpp-devel libyaml libyaml-devel libselinux-python

Install HAProxy for HTTPS

Itential Automation Gateway uses haproxy to support HTTPS security requirements. This is an optional requirement that is used alongside the HTTPS Install instructions.

yum -y install haproxy

Install Python

Automation Gateway is intentionally distributed for one particular Python installation. There are many methods to install Python and you are free to use whatever works best for your environment. As the CentOS 7 package manager does not maintain any packages after Python 3.6, you will need to perform a custom install for later versions of Python.

Example of building Python from source:

yum -y groupinstall "Development Tools"
yum -y install openssl-devel libffi-devel bzip2-devel zlib-devel wget sqlite-devel
wget https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz
tar xvf Python-3.9.2.tgz
cd Python-3.9*/
./configure --enable-optimizations --enable-loadable-sqlite-extensions
make altinstall

In this example, Python would be installed with the following significant locations:

$ which python3.9
/usr/local/bin/python3.9

$ python3.9 -m pip -V
pip 20.2.3 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

Create Non Root User

While your own user setup will be very specific to your environment and business rules, at the least we recommend creating a non root user, such as itential, to run the application. Since the Automation Gateway enables a wide variety of system administrator oriented tools, you should never run the process as root since this gives web users the ability to potentially perform system administration tasks as root on the webserver itself, instead of the intended targets (remote devices) with the intended authorization (provided credentials).

In this example, only the default user configuration is used. However, considerations such as password authentication, selinux settings, etc. should also be taken into account.

useradd itential

Create and Assign Directories

As a webserver and a gateway to other tools, Automation Gateway requires multiple directories to function properly.

Main Package Locations

As a Python application, Automation Gateway will be installed within the site-packages directory of your python installation.

• Files within this directory structure should not be modified or manipulated.
• Any updates to or API interactions with Automation Gateway could overwrite the files within this directory.
• To supply additional content (e.g., third party or custom modules and scripts) please see Integration Content Locations below.

Examples

System installed Python, Ansible installed globally:

/usr/local/lib/python3.9/site-packages/automation_gateway

Manually installed Python, Ansible installed by the itential user:

/home/itential/python3.9/site-packages/automation_gateway

Any installation method for Python, Ansible installed inside of a virtualenv at /home/itential/automation-gateway:

/home/itential/automation-gateway/lib/python3.9/site-packages/automation_gateway

Integration Content Locations

In general, there are up to three main file locations to understand for each installed integration.

• The "out of the box" content path. For Ansible, an example would be where the Ansible modules are installed by default.
• The "dynamically created" content path. For Ansible, an example would be where ansible-galaxy will install collections when told to via API calls to Automation Gateway.
• The "user supplied" content path. This is the path where you intend to supply content such as third party or custom Ansible/Nornir/Terraform modules, etc.

Careful consideration of these path types will make configuring a robust properties.yml file much easier.

Database Directory

In this example, we configure only the required directory which will hold the database files.

⚠ This database creation step is not required if using the automation-gateway-setup.sh script.

This path is required to:

• Match what is specified in properties.yml or the corresponding environment variable.
• Be accessible by the non-root user which is running the Automation Gateway process.

# Example assumes data_file=/opt/automation-gateway/data/automation-gateway.db
mkdir -p /opt/automation-gateway/data
chown itential:itential /opt/automation-gateway/data

Custom Content Example

In this example, we configure the directories which will hold the custom content files. This could be any supported content such as an Ansible collection or a Nornir configuration file.

⚠ The custom content steps are not required if using the automation-gateway-setup.sh script.

These paths are required to:

• Match what is specified in properties.yml or the corresponding environment variables.
• Be accessible by the non-root user which is running the Automation Gateway process.

# Example assumes each integration's properties.yml paths reflect the below paths
mkdir -p /usr/share/ansible/inventory \
         /usr/share/ansible/modules \
         /usr/share/ansible/roles \
         /usr/share/ansible/collections \
         /usr/share/ansible/playbooks \
         /usr/share/nornir/modules \
         /usr/share/scripts \
         /usr/share/terraform

chown -R itential:itential /usr/share/ansible \
                           /usr/share/nornir \
                           /usr/share/scripts \
                           /usr/share/terraform

Expose Ports

While mostly out of scope for this documentation, it is highly likely that you will need to configure your firewalls to allow TCP traffic on the desired socket (see the properties.yml section of Automation Gateway).

In this example, we expose the default port via the built-in CentOS 7 firewall daemon.

firewall-cmd --zone=public --add-port=8083/tcp --permanent
firewall-cmd --reload

Continue As Non Root User

The remaining installation steps should be performed as the non root user and from the non root user home directory.

In the examples that follow, the itential user is used.

echo "## SWITCH USER ##" && \
su itential
cd

Create Python Virtual Environment

Installing Automation Gateway via the python virtualenv module makes it very easy to retry a fresh installation if you make any mistakes. However, it should be understood that you can run Automation Gateway via global, user, or venv installations as you see fit for your environment.

In this example, we use the default venv module from Python but you are free to use your own tooling as required.

python3.9 -m venv $HOME/automation-gateway
source $HOME/automation-gateway/bin/activate

Due to the intended use of virtualenv, there will be no additional reminders after this. However, do not forget to source first in any new terminal sessions! For example: source $HOME/automation-gateway/bin/activate.

Install Python Dependencies

As a general practice, you should update a few main python packages prior to performing pip installs.

Install	Description
pip	Helps resolve packages and wheels in the most up to date fashion.
setuptools	Used to ensure additional package installations (e.g., automation-gateway[integrations]) goes smoothly.
wheel	Ensures that as many precompiled packages are used as possible instead of defaulting to legacy installers.

python3.9 -m pip install --upgrade pip setuptools wheel

As of pip>20.2.4, package installation strictly enforces PEP440 compliance, therefore non-compliant package versions are unable to be installed. Install pip<=20.2.4 if you run into issues resolving package versions for the automation-gateway package.

python3.9 -m pip install --upgrade pip==20.2.4 setuptools wheel

Install Automation Gateway

Beginning with release version 2020.2.18, the Automation Gateway package comes with a list of recommended integrations and their related versions. If specific integrations and versions are required, or have already been installed, use Automation Gateway Only in the next section.

For either path, you will need to know which Automation Gateway version you will be using, as well as the repository credentials (contact Itential Product Support for credentials).

export AUTOMATION_GATEWAY_VERSION=""
export REGISTRY_USERNAME=""
export REGISTRY_PASSWORD=""
export REGISTRY_URL="registry.aws.itential.com/repository/automation-gateway-release/simple"

Automation Gateway Only

This command will install only Automation Gateway and require manual installation of integrations such as Ansible, Terraform, etc.

python3.9 -m pip install \
  automation-gateway==$AUTOMATION_GATEWAY_VERSION \
  --extra-index-url https://$REGISTRY_USERNAME:$REGISTRY_PASSWORD@$REGISTRY_URL

Install with Integrations

This command will install Automation Gateway and all current recommended versions of supported integrations. The only exception is that Terraform will need to be installed manually. See the Terraform Installation example below.

python3.9 -m pip install \
  automation-gateway[integrations]==$AUTOMATION_GATEWAY_VERSION \
  --extra-index-url https://$REGISTRY_USERNAME:$REGISTRY_PASSWORD@$REGISTRY_URL

Install Separate Integrations

⚠ This step is not required if using the Install with Integrations step above.

Most Automation Gateway integrations can be installed in a variety of ways. Where possible, we recommend performing pip installations in the same environment as your Automation Gateway package to ensure they are properly discovered. This is not applicable for Terraform as it is not a pip package.

Example: Integrations via pip

python3.9 -m pip install ansible
python3.9 -m pip install nornir
python3.9 -m pip install netmiko

Example: Terraform Installation

mkdir -p $HOME/terraform
cd $HOME/terraform
curl -sL https://releases.hashicorp.com/terraform/0.12.28/terraform_0.12.28_linux_amd64.zip > terraform_0.12.28_linux_amd64.zip
unzip terraform_0.12.28_linux_amd64.zip
cp terraform /usr/local/bin

Further Reading:

Refer to the official Ansible Installation Guide for your particular integration and version combination.

Start Automation Gateway

Automation Gateway can be started on demand via the automation-gateway binary, or installed and enabled as a service via a systemd service script.

Quick Start (On Demand)

Do not forget to source if using a virtual environment. For example: source $HOME/automation-gateway/bin/activate.

automation-gateway

Service Start (Systemd Script)

Automation Gateway provides a setup script in the same location as the automation-gateway binary. In some situations, the script is in the user's path and can be found automatically via the which command.

which automation-gateway-setup.sh

However, if you are not logged in as the correct user, or if you have not sourced into your virtualenv, then which will not work or it will point to the wrong binaries. You can find all binaries on your system with the find command instead, and choose the one relevant to your installation.

find / -name "automation-gateway-setup.sh" 2>/dev/null

Once the correct binary path is determined, you can run the script either as root or with sudo.

sudo /path/to/automation-gateway-setup.sh --help

The setup script takes the following arguments, which can be found via the --help command above.

Argument	Description
--setup	This signifies to setup Automation Gateway. Mutually exclusive with clean.
--clean	This signifies to clean Automation Gateway. Mutually exclusive with setup.
--install-dir=*	This optional argument is used to create the install location of Automation Gateway. Replace the '*' with the desired install directory. The path can be absolute or relative. Used with setup and clean.
--custom-content-dir=*	This optional argument is used to create the locations for custom content (modules, etc). Replace the '*' with the desired role directory. The path can be absolute or relative. Used with setup.
--user=*	This optional argument is used to create the owner of the Automation Gateway process. Replace the '*' with the custom user. Used with setup.
--systemd	This optional argument, if passed in, signifies for the Automation Gateway process to be run as a systemd service. Used with setup.
--start-on-boot	This optional argument, if passed in, signifies that Automation Gateway will automatically start on system boot up. Systemd argument also must be passed in for this argument to work. Used with setup.
--force	This argument signifies to override confirmation statements during the --clean argument. Used with clean.

If any arguments are not passed in, the user will be prompted to input them or use defaults.

An example call for setup (explicitly uses defaults):

echo "## START AUTOMATION GATEWAY ##" && \
/path/to/automation-gateway-setup.sh \
  --setup \
  --user=itential \
  --custom-content-dir=/usr/share \
  --install-dir=/opt/automation-gateway \
  --systemd \
  --start-on-boot

After setup, follow the post installation instructions for next steps (e.g., modify the properties file that is generated with any additional changes needed for your environment).

Custom Service Environments

If your network requires using a proxy for http or https access, you may set the http_proxy and https_proxy settings via the systemd unit file. This method will properly set the environment variables for IAG.

Below is an example of incorporating environment variables for an IAG SystemD proxy modification.

...
[Service]
...
Environment=HTTP_PROXY=http://192.168.11.50:3128

Environment=HTTPS_PROXY=http://192.168.11.50:3128

Remove Service

In order to undo the setup script (remove service, directories, user, etc.), you can use the --clean argument.

An example call for clean:

/path/to/automation-gateway-setup.sh \
  --force \
  --clean \
  --install-dir=/opt/automation-gateway

Validate the Service Installation

Utilize the systemctl status and restart commands as required to validate and reload the service.

Helpful Commands

Check the status of the service

systemctl status automation-gateway

Example output:

systemctl status automation-gateway
● automation-gateway.service - Itential Automation Gateway
   Loaded: loaded (/etc/systemd/system/automation-gateway.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2021-05-20 18:25:55 EDT; 1s ago
 Main PID: 5343 (automation-gate)
   CGroup: /system.slice/automation-gateway.service
           └─5343 /home/itential/automation-gateway/bin/python3.9 /home/itential/automation-gateway/bin/automation-gateway --properties-file=/opt/automation-gateway/conf/properties.yml

May 20 18:25:55 localhost.localdomain systemd[1]: Stopped Itential Automation Gateway.
May 20 18:25:55 localhost.localdomain systemd[1]: Started Itential Automation Gateway.
May 20 18:25:56 localhost.localdomain automation-gateway[5343]: {"time": "2021-05-20 18:25:56,192", "severity": "INFO", "src": "automation_gateway.automation_gateway", "msg": "Automation Gateway starting."}
...

Check if the service has been "enabled" (will continue after a system reboot)

systemctl is-enabled automation-gateway

Reload service and restart Automation Gateway after service changes

systemctl daemon-reload
systemctl restart automation-gateway

Restart Automation Gateway after properties.yml changes

systemctl restart automation-gateway

Restart Automation Gateway and track the server logs via journalctl

systemctl restart automation-gateway && journalctl -f -u automation-gateway

Disable Automation Gateway from persisting through a reboot

systemctl disable automation-gateway

Stop Automation Gateway

systemctl stop automation-gateway

Next Steps

Once you have successfully started Automation Gateway, you will want to ensure your properties.yml and integrations are working as expected. See the properties.yml section of Automation Gateway for additional configuration details. See the Operations section for additional details on testing and utilizing Automation Gateway.