Back up and restore MongoDB

This page describes how to back up and restore the MongoDB database used by Itential Platform. The following strategy minimizes downtime, maximizes efficiency, and helps prevent service level agreement (SLA) breaches.

Backup strategy

Itential recommends these backup practices:

Perform a daily mongodump to capture all database data once per day.
Take hourly snapshots of the jobs and tasks collections to capture automation results throughout the day.
Retain backups for no longer than seven days, as backup files can become large.

You can capture hourly snapshots in binary format or JSON format. Itential prefers binary format and recommends using it whenever possible.

Identify critical directories and files

Ensure you identify and back up the following files before restoration. The locations below are defaults and may vary in your installation.

File type	Location	Description
Configuration files	`/etc/mongod.conf`	Main configuration
Security and authentication files	`/etc/ssl/certs/mongo`	TLS certificates and keys
Security and authentication files	`/etc/ssl/`	Additional security files
Data directory	`/var/lib/mongo`	Database files

Back up MongoDB data

You can only export and import from the primary MongoDB server in the replica set. Connect to the primary server, then use mongoexport or mongodump to export content to the appropriate directory. Move the data to the target server and use mongoimport or mongorestore to recover from these backups.

For complete MongoDB documentation, see the MongoDB backup and restore tools guide.

Back up the entire database

Export the database

Export the database to a ./dump/ directory using the hostname of the current primary server in the replica set:

$ mongodump --db=itential --oplog <other tls options> --uri='mongodb://admin:<password>@<mongo-host1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin&readPreference=secondary'

Package and transfer the backup

$ tar -czvf mongodump-20230906.tgz dump
$ scp mongodump-20230906.tgz <user>@<target-server>:<target-directory>

Back up a specific collection

Use mongodump for binary backups or mongoexport for human-readable JSON format.

Binary format

Export to a binary file

The mongodump command below dumps the workflows collection to a dump directory. You can run this directly against a secondary server or connect to the replica set and prefer the secondary:

$ mongodump --ssl --collection=workflows --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin&readPreference=secondary'

Package and transfer the backup

$ tar -czvf mongodump-20230906.tgz dump
$ scp mongodump-20230906.tgz <user>@<target-server>:<target-directory>

JSON format

Export to a JSON file

You can use --ssl or --tls options:

$ mongoexport --ssl --db=itential --collection=workflows --out=workflows/workflows.json --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin&readPreference=secondary'

Package and transfer the backup

$ tar -czvf workflows.json.tgz workflows/*
$ scp workflows.json.tgz <user>@<target-server>:<target-directory>

Restore the data

When you restore a member of a replica set, MongoDB ensures that the new member doesn’t become the primary until it syncs with the existing replica set. MongoDB controls this automatically through member states, but you can also manually control whether a member can become primary by setting its priority.

Member states

If you add a new member to a replica set, or if an existing member has been out of service for an extended period, MongoDB initiates a series of steps to bring that member in line with the existing members.

STARTUP

Each replica set member starts in this state.
mongod loads the member’s replica set configuration and transitions the member to STARTUP2.
Members cannot vote and are not recognized as replica set members yet.

STARTUP2

Members enter this state after mongod finishes loading the configuration.
The member becomes an active replica set member.
If the member begins initial sync, it remains in STARTUP2 until all data is copied and all indexes are built, then transitions to RECOVERING.

RECOVERING

Members are not ready to accept reads.
This state can occur during normal operation and does not indicate an error.
Members can vote in elections but cannot become primary.
Members transition to SECONDARY after replicating enough data for consistent client reads. The only difference between RECOVERING and SECONDARY is that RECOVERING prohibits client reads while SECONDARY permits them.

Control member priority

To prevent a secondary member from becoming primary during failover, assign it a priority lower than the primary member’s priority.

Example: Your current primary is m1.example.net and you want to make m3.example.net the primary. You have a three-member replica set with the following configuration:

1 {
2   "_id": "rs0",
3   "version": 7,
4   "members": [
5     { "_id": 0, "host": "m1.example.net:27017" },
6     { "_id": 1, "host": "m2.example.net:27017" },
7     { "_id": 2, "host": "m3.example.net:27017" }
8   ]
9 }

Update member priorities

In a mongo shell connected to the primary, run:

$ cfg = rs.conf();
$ cfg.members[0].priority = 5;
$ cfg.members[1].priority = 5;
$ cfg.members[2].priority = 10;
$ rs.reconfig(cfg);

Verify the election

The following sequence occurs automatically:

m3.example.net and m2.example.net sync with m1.example.net (typically within 10 seconds).
m1.example.net detects it no longer has the highest priority and steps down. If m3.example.net’s sync is far behind, m1.example.net waits until m3.example.net is within 10 seconds of its optime before stepping down.
The step-down forces an election where m3.example.net becomes primary based on its priority setting.

Complete the restoration

Use the appropriate restore command based on your backup method.

Restore from binary format

If you used mongodump to create the backup, use mongorestore to restore it:

$ mongorestore --ssl --drop --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin' dump/itential

Restore from JSON format

If you used mongoexport to create the backup, use mongoimport to restore it:

$ mongoimport --ssl --db=itential --collection=workflows --file=workflows/workflows.json --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin'

For more information, see the MongoDB replica set restoration documentation.

Next steps

After completing the backup and recovery process, consider these additional measures:

Set up automated backup monitoring to ensure backups complete successfully.
Test your recovery procedures regularly to verify they work as expected.
Document your specific configuration details for future reference.
Consider implementing additional security measures for backup storage.

Learn more

Backup strategy

Itential recommends these backup practices:

Perform a daily mongodump to capture all database data once per day.
Take hourly snapshots of the jobs and tasks collections to capture automation results throughout the day.
Retain backups for no longer than seven days, as backup files can become large.

You can capture hourly snapshots in binary format or JSON format. Itential prefers binary format and recommends using it whenever possible.

Identify critical directories and files

Ensure you identify and back up the following files before restoration. The locations below are defaults and may vary in your installation.

File type	Location	Description
Configuration files	`/etc/mongod.conf`	Main configuration
Security and authentication files	`/etc/ssl/certs/mongo`	TLS certificates and keys
Security and authentication files	`/etc/ssl/`	Additional security files
Data directory	`/var/lib/mongo`	Database files

Back up MongoDB data

For complete MongoDB documentation, see the MongoDB backup and restore tools guide.

Back up the entire database

Export the database

Export the database to a ./dump/ directory using the hostname of the current primary server in the replica set:

$ mongodump --db=itential --oplog <other tls options> --uri='mongodb://admin:<password>@<mongo-host1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin&readPreference=secondary'

Package and transfer the backup

$ tar -czvf mongodump-20230906.tgz dump
$ scp mongodump-20230906.tgz <user>@<target-server>:<target-directory>

Back up a specific collection

Use mongodump for binary backups or mongoexport for human-readable JSON format.

Binary format

Export to a binary file

The mongodump command below dumps the workflows collection to a dump directory. You can run this directly against a secondary server or connect to the replica set and prefer the secondary:

$ mongodump --ssl --collection=workflows --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin&readPreference=secondary'

Package and transfer the backup

$ tar -czvf mongodump-20230906.tgz dump
$ scp mongodump-20230906.tgz <user>@<target-server>:<target-directory>

JSON format

Export to a JSON file

You can use --ssl or --tls options:

$ mongoexport --ssl --db=itential --collection=workflows --out=workflows/workflows.json --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin&readPreference=secondary'

Package and transfer the backup

$ tar -czvf workflows.json.tgz workflows/*
$ scp workflows.json.tgz <user>@<target-server>:<target-directory>

Restore the data

Member states

STARTUP

Each replica set member starts in this state.
mongod loads the member’s replica set configuration and transitions the member to STARTUP2.
Members cannot vote and are not recognized as replica set members yet.

STARTUP2

Members enter this state after mongod finishes loading the configuration.
The member becomes an active replica set member.
If the member begins initial sync, it remains in STARTUP2 until all data is copied and all indexes are built, then transitions to RECOVERING.

RECOVERING

Members are not ready to accept reads.
This state can occur during normal operation and does not indicate an error.
Members can vote in elections but cannot become primary.
Members transition to SECONDARY after replicating enough data for consistent client reads. The only difference between RECOVERING and SECONDARY is that RECOVERING prohibits client reads while SECONDARY permits them.

Control member priority

To prevent a secondary member from becoming primary during failover, assign it a priority lower than the primary member’s priority.

Example: Your current primary is m1.example.net and you want to make m3.example.net the primary. You have a three-member replica set with the following configuration:

1 {
2   "_id": "rs0",
3   "version": 7,
4   "members": [
5     { "_id": 0, "host": "m1.example.net:27017" },
6     { "_id": 1, "host": "m2.example.net:27017" },
7     { "_id": 2, "host": "m3.example.net:27017" }
8   ]
9 }

Update member priorities

In a mongo shell connected to the primary, run:

$ cfg = rs.conf();
$ cfg.members[0].priority = 5;
$ cfg.members[1].priority = 5;
$ cfg.members[2].priority = 10;
$ rs.reconfig(cfg);

Verify the election

The following sequence occurs automatically:

m3.example.net and m2.example.net sync with m1.example.net (typically within 10 seconds).
m1.example.net detects it no longer has the highest priority and steps down. If m3.example.net’s sync is far behind, m1.example.net waits until m3.example.net is within 10 seconds of its optime before stepping down.
The step-down forces an election where m3.example.net becomes primary based on its priority setting.

Complete the restoration

Use the appropriate restore command based on your backup method.

Restore from binary format

If you used mongodump to create the backup, use mongorestore to restore it:

$ mongorestore --ssl --drop --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin' dump/itential

Restore from JSON format

If you used mongoexport to create the backup, use mongoimport to restore it:

$ mongoimport --ssl --db=itential --collection=workflows --file=workflows/workflows.json --uri='mongodb://admin:<password>@<mongo-host-1>:27017,<mongo-host-2>:27017,<mongo-host-3>:27017/itential?replicaSet=rs0&authSource=admin'

For more information, see the MongoDB replica set restoration documentation.

Next steps

After completing the backup and recovery process, consider these additional measures:

Set up automated backup monitoring to ensure backups complete successfully.
Test your recovery procedures regularly to verify they work as expected.
Document your specific configuration details for future reference.
Consider implementing additional security measures for backup storage.

$	tar -czvf mongodump-20230906.tgz dump
$	scp mongodump-20230906.tgz <user>@<target-server>:<target-directory>

$	tar -czvf workflows.json.tgz workflows/*
$	scp workflows.json.tgz <user>@<target-server>:<target-directory>

1	{
2	"_id": "rs0",
3	"version": 7,
4	"members": [
5	{ "_id": 0, "host": "m1.example.net:27017" },
6	{ "_id": 1, "host": "m2.example.net:27017" },
7	{ "_id": 2, "host": "m3.example.net:27017" }
8	]
9	}

$	cfg = rs.conf();
$	cfg.members[0].priority = 5;
$	cfg.members[1].priority = 5;
$	cfg.members[2].priority = 10;
$	rs.reconfig(cfg);