In this brief blog post, we will talk about Barman cloud utilities, which greatly ease the process of storing backups on cloud platforms like GCP, AWS, Azure, etc. Backups are of paramount importance, and in PostgreSQL, we also need to retain the WAL files, which can be used for various purposes like incremental backups or some PITR/ad-hoc recoveries. These requirements can be fulfilled at an instant pace with the help of these cloud-native utilities, which we are going to discuss next.

In order to use the tools, we need to install/set them up in the Postgres environment where we plan to execute the backup operation.

Installation of Barman cloud utilities

For the demo purpose, I installed it on Red Hat 9, but it can be installed on other OS platforms as well. 

Note – For Red Hat distribution “barman-cli-cloud” package automatically included in “barman-cli” package. For other platforms like Debian/Ubuntu we need to install “barman-cli-cloud” also.

We also need to resolve certain dependencies based on the environment.

Authentication

GCP (Google Cloud Platform)

We can download a JSON-based key (gcs-key.json) from the GCP service account for the associated user/account. We also need to make sure the folder storage role and other permissions are attached to the target bucket for writing, reading, listing, and other bucket-related events.

The authentication is supported only via the environment variable “GOOGLE_APPLICATION_CREDENTIALS” as below for the GCP platform.

We can also persist the above changes by putting them under ​​ “~/.bashrc” OR “~/.bash_profile” of the “postgres“ user. 

AWS (Amazon Web Services)

For AWS, we need to install the “awscli tool” in order to set up the profile and authentication to interact with the S3/Bucket.

The details can be verified as below.

Backup

Taking a full backup is just a matter of executing the below single command. 

Note – Here “pg17” is the server name which will create the folder with the same name in the bucket as well. It’s not mandatory to use the exact name and we can use any naming here. This will be required in fetching the backup/recovery related details and information.

Output:

We have some extra parameters to enhance/control the whole backup and upload process.

The above example was taken from the GCP platform. For others, like AWS and Azure, all we need to do is replace the “–cloud-provider” option with one of the values below.

  • aws-s3
  • azure-blob-storage.
  • google-cloud-storage.

Backup status and listing

By using the “barman-cloud-backup-list” utility, we can list all the backups.

Output:

To gain more insight into the particular backup, we can use the “barman-cloud-backup-show” utility. This will show all the metadata details about the backup.

Output:

WAL archiving/backup

Preserving WAL files is an important consideration for PITR purposes and can also be used as an incremental backup.

To enable WAL archiving, we can use the “barman-cloud-wal-archive” utility in the Postgres “archive_command“ directly. This will copy the WAL files over the remote buckets.

For the changes to take effect, we need to restart the service.

Verify the configuration change:

The utility can be run manually also, especially handy for troubleshooting purposes or some quick testing.

Output:

To verify the WALs, we can either directly check the bucket or list the files from the invoking source itself.

Output:

Similarly with WAL archiving as well we have some extra options to control and better perform the archiving process.

Restoring backup

First, we need to stop the database service and empty the existing data directory “/var/lib/pgsql/17/data/”. Probably better to take a backup of the data directory before removals.

Data restoration also comes very handy with the help of “barman-cloud-restore” utility. All we need to do is pass the backup ID  “20250305T163648 “to the command.

Output:

Once the files are copied to the target data directory  “/var/lib/pgsql/17/data/,”  we need to run the recovery process to ensure a consistent restore. 

1) create the recovery signal file inside the data directory.

2) Make sure to disable or comment out the archive_command temporarily to prevent writing to the same bucket path again.

Restoring WAL

The “barman-cloud-wal-restore” utility fetches the required remote WAL file and copies it to the target “pg_wal” folder again. This works the same as the native “restore_command.“

3) Next, add the restore_command and set the “recovery_target_timeline” in “/var/lib/pgsql/17/data/postgresql.conf” file.

We can also do a quick test by running the direct “barman-cloud-wal-restore“ command.

4) Finally, start the Postgres service and verify the tables and data.

Final thoughts

Although we already have the native Barman solution for taking remote backups, with the introduction of the cloud-native tool “barman-cloud-*,”  running backups over different cloud platforms is super handy now. The recovery process itself is pretty simple. All we need to do is restore the backup + desired WAL file. This setup also eliminates the need to have a separate Barman server and a set of other configurations, as we are directly running the tools in the Postgres environment. It’s still possible to use the cloud features with the native Barman server-style setup by using various hook scripts.  

PostgreSQL Performance Tuning

Subscribe
Notify of
guest

0 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments