This post is a sequel to my previous post, which is, How to backup local VMs to the remote backup server? but in this post, we will learn to sync a local Proxmox backup server to a remote Proxmox backup server, the sync strategy is named reverse sync, and is accomplished by using two features of the Proxmox backup server, namely remotes and sync jobs.
To complete the tutorial, we will need two Proxmox backup servers, one server will be installed locally and will run on top of the Proxmox VE server as a VM and the other will be installed remotely, here are the two guides, to help you create local and remote backup servers. It is strongly suggested to go through these guides, as they will be referenced in the post.
In the second guide, I have used aws ec2 instance as the remote cloud location, you may choose any non-cloud remote location with the proxmox backup server installed.
Port forwarding 8007 Port For Local and Remote Proxmox Backup Servers
Assuming that you have gone through the guides, and both local and remote backup servers are up and running and Datastores have also been created then that completes step 1 of the tutorial, now for step 2, we need to port forward port 8007 for both local and remote backup server, I am assuming that both local and remote servers are behind a firewall and using NAT.
The cloud level port-forwarding is already discussed in the first guide but for the port-forwarding at the local site, here is a simple guide to understanding NAT and Port-forwarding.
If you have done forwarding port 8007 on both local and remote sites then step 2 is complete. For step 3, we will configure scheduled incremental backups on our local backup server, and then in step 4, we will do the same for our remote proxmox backup server.
Incremental Live VM Backups On Local Proxmox Backup Server
Although I am using the local backup server as a VM hosted on proxmox ve, instead you can also install it on a separate machine and place it on the same network as proxmox ve.
After port forwarding, we need to add the local backup server Datastore, as storage to proxmox ve, here are a few easy steps to do it. once the datastore is added then we are ready to take backups.
If you have gone through the guides then you are already familiar with backup modes; stop, suspend and snapshot, here is an example to schedule live periodic incremental backups, just make sure that you select the backup mode to be snapshot and VM should be running.
Once the Full VM backup is complete then later backups are incremental and will execute according to configured schedule, which is also shown in the above example link.
If you have reached this far then you have learned to create a schedule for incremental backups on the local proxmox backup server, and hopefully, no considerable lag in VM performance was noticed during the backup process.
Sync Backups Between Local And Remote Proxmox Backup Server
As the incremental backups or snapshots are populating according to the configured schedule on the local backup server, the same schedule and snapshots can be replicated on the remote proxmox backup server, and to do this, we will use two features, remotes, and sync job.
The Remotes Feature
Proxmox backup server offers a feature named Remotes, this allows us to add remote backup servers to our local backup server and vice versa.
For this tutorial we will not use the Remotes feature on the Local Backup server rather we will use it on the remote backup server, I will explain the reason down the post, so let’s use it on the remote server.
Since we have already allowed port 8007 on the local and remote firewalls, it’s time to add the local proxmox backup server to our remote backup server. To do this log into your local backup server, go to Dashboard –> in the right pane, select Show Fingerprint, and then copy this Fingerprint and save it somewhere, it will be used in the next step. After that go to, remote backup server and log into it, and then from the left pane select Remotes—> then select Add button which will open a Form;
In this form, we are going to add the Local backup server information as follows.
Remote: Enter the name for the local backup server.
Host: Enter the Public Static IP address of your local backup server.
Auth ID: This will be root@pam by default unless you have changed or created the new username.
Password: Enter the password for the root, if you have created a new user then you will enter the password for the new user.
Fingerprint: Paste the fingerprint that you saved in the previous step.
Comment: That is up to you.
Once the information is entered, select the Add button and then the local backup server is added to your remote backup server.
Remotes and the Sync Job (Reverse backup sync)
You may have been thinking that why I have started in the reverse order? shouldn’t I add the remote backup server to the local server first? and then push or sync local backups to the remote backup server.
The answer is that the proxmox backup server does not offer the functionality to push backups, it only offers the functionality to pull, and this functionality is called sync job.
So the local backup server can only pull backup data from the remote backup server but cannot push VM backups, so as a workaround, we need to configure the remote backup server to pull the backup data from our local backup server using the sync job, hence both remotes and sync job are configured on the remote backup server.
This post solves the same problem which is discussed in these two threads:
- Reverse direction of Proxmox Backup Server remote sync
- Allow a user to push his datastore to a remote server
Configure Sync Job on the remote backup server
As we are already taking incremental VM backups on our local backup server, it’s time that these backups are pulled by the remote backup server, and we may also refer to it as reverse sync.
As in the previous step, on the remote backup server, we have added the local backup server, now to start the reverse sync process and check whether adding the local server was a success? on the remote server, move to the left pane —> select previously created Datastore —> then from the right pane select Sync Jobs—>then select Add, this will open the sync job form.
If you see the local backup server information then the Source Datastore field will show the related Datastore.
After selecting the Datastore just add the desired sync schedule, let’s say hourly, the result is that the remote backup server will pull the most recent snapshot from the local backup server every hour.
Removed Vanished: If you select the check box then whenever you remove the snapshot from the local backup server, the corresponding snapshot will also be removed from the remote backup server, I would recommend leaving it at the default, uncheck state.
local Incremental Backup synced with remote backup server
let us now see the whole process in action, This demo executes a live VM incremental backup, the VM name is 100, and it will be backed up on the local backup server, this local backup server is also running as a VM on another Proxmox VE.
At first, a backup job with a schedule is created for VM100, when the schedule executes, the first backup is always a full VM backup and the later backups are incremental.
The screenshot shows, creating a backup job and setting a schedule for VM 100:
Although these settings have been described in the above-recommended guides already but still here is a brief intro to these settings:
Node: represents pve host, storage: represents the pbs datastore added as storage to pve, Day of the week: you can choose a single day or whole week, Start Time: it is the time when the schedule will start to execute, Selection Mode: let it be at the default value, Send email to: you need to configure SMTP server so that notifications can be sent to you on failure, Email notification: set it to on failure only, Compression: zstd is the best and the fastest compression, Mode: It must be set to snapshot in case of incremental backups, Enable: checkbox should be checked to enable the job and vice versa.
After this hit the Create button to create the backup job. You can also manually start the job before the scheduled time, by selecting the Run Now button. Once the job is started, the first backup will be the full VM backup and later backups are incremental.
As you can see a backup job has started for VM100.
Here is the further detail for the backup job in the Task viewer.
Here is the incremental backup job executing on the local backup server.
You can also see the snapshot date & time and also the snapshot count by selecting your Datastore—> Content.
If you select the highlighted + sign then it will show the detail for every snapshot.
While these local incremental backups are executing on the local backup server, a sync job is configured on the remote proxmox backup server.
To set up the sync job on the remote backup server, at first, the remotes feature is configured and then the Sync job feature is configured, The screenshots for these two features have already been shared in the above section. Once the Sync job is configured the local backup will be pulled by the remote backup server.
The good thing about sync jobs is that it keeps account of the number of new snapshots created between two subsequent syncs.
You can also check the snapshot detail such as snapshot/backup time and count of the snapshots on the remote proxmox backup server, in the same way as in the case of the local backup server.
Here is a screenshot showing sync status, it also shows the last sync time, the time it took to sync, and the next time the sync job will run.
After the successful sync job, you will also see the same snapshots on the remote & local backup server.
This brings us to the end of this post, as incremental backups for live VMs are being reverse synced with a remote backup server, at the desired schedule.
The Topics That Are Not Covered In The Post
Here is a list of topics that are closely related to this Tutorial, each topic demands a separate independent post and I shall be writing about them very soon, IA.