I Wiped My Proxmox Host to See If This Backup Tool Could Bring It Back

Are you truly protected against data loss in your Proxmox self-hosting environment? While most disaster recovery plans focus on virtual machine and LXC container backups, a critical piece often goes overlooked: the Proxmox VE Server host configuration itself. Imagine losing your intricate network setups or cluster configurations! This article dives deep into the often-missed necessity of Proxmox host backup, exploring how ProxMenux steps in to fill this vital gap. We’ll put its host backup and restore functionality to the test in a real-world, complex cluster scenario, revealing its strengths and identifying areas where manual intervention might still be needed for a robust Proxmox disaster recovery strategy.

Beyond VMs: The Overlooked Proxmox Host Backup in Disaster Recovery

A critical missing piece in most disaster recovery (DR) strategies for Proxmox environments often lies with the hosts themselves. We instinctively prioritize backing up virtual machines (VMs) and LXC containers, which house workloads and data – undoubtedly a crucial component. However, the protection of the actual Proxmox VE Servers often falls by the wayside, a gap that has fortunately started to gain more recognition. Tools like ProxMenux, which I’ve used extensively in my home lab, now include robust Proxmox VE Server host configuration backups. Let’s explore how these are configured and evaluate their effectiveness.

The Proxmox server itself frequently represents a backup blind spot. This isn’t unique to Proxmox; in the VMware ESXi world, many vSphere admins overlook backing up their ESXi host configurations. While excellent out-of-the-box features in Proxmox ensure good coverage for guest workloads like VMs, LXCs, and Docker managed volumes, the underlying physical host is a distinct entity. A significant amount of configuration, particularly on the cluster and networking side, resides on a Proxmox host – data you absolutely wouldn’t want to lose.

Unless you’re leveraging Proxmox SDN with a centralized management plane for your networking, most setups rely on traditional Linux bridges. Losing that meticulously crafted network configuration, let alone other host settings, could mean hours of painstaking recreation.

Comprehensive Proxmox Recovery: Two Critical Layers

For true Proxmox disaster recovery, your plan should encompass two fundamental layers:

• The Proxmox host, including its operating configuration and installed components.
• The VMs and LXC containers running on the host.

ProxMenux: Your Solution for Proxmox Host Configuration Backup

ProxMenux is a powerful solution I’ve covered previously and actively utilize in my home lab. It offers excellent functionality across the board, but a less-known feature added in recent releases is its Host Backup & Restore capability.

This functionality directly addresses the first layer of data protection, safeguarding your physical host should disaster strike. Beyond flexible scheduling, ProxMenux boasts the excellent addition of targeting Proxmox Backup Server (PBS) as a destination. Many configuration backup tools require local storage or a generic file share, so having all your protected files and DR resources consolidated within PBS is a significant advantage.

ProxMenux also allows you to restore the backup to a clean Proxmox installation. This process applies the saved configuration to your new Proxmox setup, meaning you’re restoring “settings” rather than a full “image” of the server. This method is inherently safer, especially when dealing with clustered environments, as it avoids potential issues with hardware differences or stale image data.

Deconstructing ProxMenux Host Backups: What’s Included?

So, what exactly does a ProxMenux host backup capture? It includes three crucial components of your host’s overall configuration:

• Filesystem data
• Structured manifest
• Application inventory

Below is a breakdown of what each component includes and why it’s vital:

As you can see, the comprehensive nature of these components and their backup methodology makes this approach significantly safer and more resilient than simply restoring a generic server image. Your recovery effectively becomes a proper reconstruction of the essential data needed to bring your PVE host back to a functional state.

Putting ProxMenux to the Test: A Real-World Proxmox Cluster Scenario

Before enthusiastically recommending this tool for your “production” self-hosting environments, I conducted a rigorous test. My setup for this evaluation consisted of:

• Three nested Proxmox VE Server hosts, fully configured.
• All hosts running the latest Proxmox 9.2.20 updates.
• A clustered environment.
• Shared storage targeting a nested TrueNAS server.

This cluster, which I previously used to test the TrueNAS plugin, represents a challenging “worst-case” scenario due to the hosts requiring the TrueNAS plugin and having existing shared storage configurations. My goal was to determine if a ProxMenux host backup could successfully restore one of these complex cluster nodes by simply introducing a clean Proxmox VE server and applying the configuration.

Integrating Proxmox Backup Server (PBS) as a Backup Destination

My first step was to add my PBS storage to the Proxmox VE Cluster, ensuring it was available as a target for the ProxMenux backup.

Initiating Your Proxmox Host Backup with ProxMenux

Leveraging nested virtualization, creating rollback points was simple using snapshots. I took a snapshot of all three hosts, named “before-wiping-out-pvetest01-9-23-2026,” to safeguard the cluster before modifying pvetest01. This flexibility is a key advantage of nested labs for learning and testing.

To get started, you’ll need to install ProxMenux. For a quick installation, execute the following command from your Proxmox VE Server command line:

bash -c "$(wget -qLO - <https://github.com/DerDanilo/proxmenux/raw/main/proxmenux_install.sh>)"

Once installed, you can access the Host Backup & Restore functionality via the terminal by navigating to: menu > Host Backup & Restore. Alternatively, the same backend is accessible from the “Backups” tab in the ProxMenux Monitor web interface (defaulting to http://:8008). The web interface offers additional features like live logging, notifications, and a rollback delta view, while the terminal workflow is ideal for SSH and unattended operations.

After configuring your PBS storage in Proxmox’s Datacenter > Storage settings, you’ll see it listed as a destination within ProxMenux. My clpbs storage was already available, ready for use. To create the backup, I initially used the TUI interface. Under “Host Backup & Restore,” I selected “Backup host configuration,” then “Backup to Proxmox Backup Server (PBS),” confirming my PBS connection and choosing to encrypt the backup. I accepted the auto-populated backup ID, and the process began, successfully completing the backup.

The ProxMenux web interface then clearly displayed the newly created backup, offering options to restore, download, or view its contents, including the detailed breakdown of the backed-up configuration files.

The Ultimate Test: Restoring a Proxmox Cluster Node from Scratch

Creating a backup is only half the battle; the true test lies in the restore. To push ProxMenux to its limits, I performed a full reinstall of pvetest01 using the Proxmox 9.1 ISO. I even chose a different hostname during the installation, aiming to see if the restore would revert it along with the networking. Logging into pvetest02 immediately showed pvetest01 as “down,” as expected.

This simulated a catastrophic OS failure, requiring a complete wipe and fresh installation. The objective was to see if the ProxMenux backup could seamlessly bring this new installation back into the cluster, with all previous configurations intact.

Restoring the Host Configuration: Initial Steps and Hurdles

The restore process began with a few hurdles. After reloading the host, I reinstalled ProxMenux and then manually added the PBS storage through the Proxmox web UI on the newly installed host. Navigating to the ProxMenux web interface, I connected to PBS and initiated the restore, opting for a full restore. The system confirmed the process, beginning with “safe path” files that could be applied live. It then prompted for a reboot to apply the remaining configurations.

Upon reboot, a critical disappointment emerged: the restore did not revert the Proxmox VE Server’s hostname or IP address. I manually corrected these to match the original configuration and re-attempted the ProxMenux restore. However, even after this, the cluster status remained problematic.

Troubleshooting Cluster Reintegration: Manual Steps Required

Despite ProxMenux reporting a successful restore, the rebuilt host remained isolated from the rest of the cluster. While the /etc/pve/corosync.conf file was correctly restored with all three nodes listed, the Corosync service was inactive, reporting that /etc/corosync/corosync.conf was missing.

It appeared ProxMenux had restored the cluster database but not the Corosync configuration symlink or the necessary authentication key. I had to manually intervene:

mkdir -p /etc/corosync
ln -s /etc/pve/corosync.conf /etc/corosync/corosync.conf
scp [email protected]:/etc/corosync/authkey /etc/corosync/authkey
chmod 0400 /etc/corosync/authkey
chown root:root /etc/corosync/authkey
systemctl start corosync

After creating the symlink and copying the authkey from a healthy node, Corosync finally recognized all three cluster members and established quorum. I then manually restarted the pve-cluster service.

Another issue arose: the TrueNAS plugin was missing. I’m unsure why the application inventory didn’t capture this manually installed APT package. Reinstalling the TrueNAS plugin and performing one final reboot brought the host back online, healthy within the cluster, and with shared storage fully operational.

Conclusion: ProxMenux for Your Self-Hosting Lab – Strengths and Areas for Improvement

In conclusion, ProxMenux is an excellent all-around tool, especially for the home lab. However, this real-world test uncovered some specific limitations in its host backup and restore process for complex scenarios. While it captured most configurations, tackling a downed Proxmox cluster node with shared storage and third-party plugins (like TrueNAS) required some manual post-restore “hacking” to fully reintegrate the host. This included creating missing Corosync symlinks and keys, and reinstalling the TrueNAS plugin.

Despite these challenges, I highly recommend ProxMenux for self-hosting environments. It offers invaluable protection for Proxmox host configurations. This particular aspect of the tool, however, could benefit from further polish, especially concerning cluster awareness and a more comprehensive application inventory. I suspect a restore of a single, non-clustered host would proceed much more smoothly. Have you tested this functionality in a similar setup? I’d be interested to hear if your results differed!

FAQ

Question 1: Why is Proxmox host backup so important for self-hosting?

Answer 1: For any serious self-hosting setup, a Proxmox host backup is critical because it safeguards the foundational configuration of your virtualization server. This includes intricate network settings (like custom bridges or VLANs), cluster configurations (Corosync, shared storage mounts), and essential system services. Losing this data would mean hours, if not days, of manual re-configuration, making proper host backups as vital as your VM backups for efficient Proxmox disaster recovery.

Question 2: Does ProxMenux restore an “image” or “settings” of the Proxmox host?

Answer 2: ProxMenux performs a “settings” restore rather than an “image” restore. This means it reconstructs the critical files and configurations on a freshly installed Proxmox system. This method is generally safer and more flexible than deploying a full disk image, especially in clustered environments or when migrating to slightly different hardware, as it avoids potential inconsistencies or outdated system components that might come with an older image.

Question 3: What are some limitations of ProxMenux host backup in complex Proxmox clusters?

Answer 3: While ProxMenux is highly effective for basic and single-node Proxmox host backups, its limitations in complex cluster scenarios, as demonstrated in our test, include not fully restoring certain cluster-specific components like Corosync symlinks or authentication keys, and potentially missing some third-party plugins (e.g., TrueNAS integration) from its application inventory. This means manual intervention might still be necessary post-restore to fully reintegrate a node into a production-level Proxmox cluster. Always complement automated backups with thorough documentation of your unique cluster setup and any manual post-restore steps.

Read the original article