Virtualization with Proxmox 8.4: Direct access from VMs to the host file system

Contents

Proxmox Virtual Environment (VE) 8.4 brings many small improvements to details as well as three interesting new functions for administrators: direct access from VMs to the host file system without the detour via network protocols, the uninterrupted moving of VMs with vGPU support to a Proxmox host in live operation and the integration of any backup solutions into the Proxmox infrastructure.

Proxmox VE 8.4 again roughly follows the semi-annual update cycle and thus updates version 8.3 from November last year. The enterprise virtualization package is now based on Debian GNU/Linux “Bookworm” version 12.10 and uses its Linux kernel 6.8.12-9 as standard. As an option, the Proxmox developers offer Linux kernel 6.14, which includes important bug fixes for AMD Secure Nestated Pageing (AMD SNP) and Uncached Buffered I/O for fast storage devices.

iX-Workshop: Setting up a Proxmox VE cluster with Ceph storage system

In this hands-on workshop, IT administrators will learn how to set up and manage a high-availability Proxmox VE cluster in conjunction with a Ceph storage system. They will familiarize themselves with the basic concepts, best practices and troubleshooting methods.

Registration and dates at heise.de/s/ppABK

The other core components of the open source system have also been updated: QEMU 9.2 with extended VirtIO GPU support for Vulkan applications is responsible for emulating and virtualizing hardware. As with Proxmox VE 8.2 and 8.3, LXC 6.0 provides the Linux containers – nothing worth mentioning has changed in LXC in over a year, apart from a handful of bug fixes.

The standard file system OpenZFS has been updated from version 2.2.6 to 2.2.7 and thus directly supports Linux kernels up to 6.12. The Proxmox developers provide the necessary patches for the optional Linux kernel 6.14. Unfortunately, the OpenZFS 2.3 branch from January of this year did not make it into Proxmox VE 8.4. Proxmox users will therefore have to wait a little longer for RAIDZ expansion, fast deduplication and direct I/O through the ARC. Thomas Lamprecht explained this decision in the Proxmox forum: Since the ZFS release 2.3 brings some profound changes – especially in the handling of the ARC (Adaptive Replacement Cache) and its interaction with the kernel, for example by marking the memory as reclaimable –, the developers have decided not to integrate ZFS 2.3 into Proxmox VE 8.4 for the time being, but to let the OpenZFS innovations “mature” a bit as a precaution.

virtiofs: VM accesses host directory directly

Thanks to virtiofs, the VirtIO file system, VMs under Proxmox VE 8.4 can access parts of the host file system directly without the detour via a network protocol. virtiofs uses the logical proximity of the VMs to the hypervisor and offers significantly higher performance than virtio-9p, for example, which runs via the network stack. It is implemented via the virtiofsd daemon, which is called directly before QEMU is started. Linux guest systems with kernel 5.4 or higher support this function by default. A corresponding driver is required for Windows guests.

One should perhaps not fall into blind euphoria because virtiofs ultimately uses FUSE (Filesystem in Userspace) as a foundation, but does not run parts of it in the relatively secure userspace, but directly on the host. Unlike persistent NFS connections, for example, a crash of virtiofs currently also requires a restart of all VMs involved. Memory hotplug and live migration should also not work together with virtiofs. Nevertheless, virtiofs should be an exciting new solution for VMs that need to quickly access the host's local data directories. Virtiofs will probably not work with Linux containers (LXC) in the foreseeable future, as it is tailored to QEMU. This is not necessary because directories outside a chroot environment or a container can also be integrated via bind-mounts –, for example. Not nice, but quick and easy.

iX Newsletter: Exciting background information on the new issue every month

Do you already know about the free iX newsletter? Register now and don't miss anything on the monthly publication date: heise.de/s/NY1E The next issue will be about the cover topic of the May iX: How to be attacked without malware.

Proxmox has long supported the live migration of VMs, and thanks to SSH tunnels and encryption, this is of course also secure. It has also been possible to use “Mediated Devices” and thus vGPUs since Proxmox VE 5.3 (2018). Proxmox VE 8.3 brought adjustments to the 6.8 kernel, and together with Nvidia's vGPU software 18, the new Proxmox VE 8.4 is now an officially supported platform. The pve-nvidia-vgpu-helper tool helps with the setup. Proxmox VE 8.4 can migrate these mediated devices – currently only works with Nvidia vGPUs – together with the running VM in the Proxmox cluster.

Farewell to Ceph 17, Bring Your Backup Solution

Proxmox VE 8.4 introduces support for external backup providers and thus extends the flexibility of backup management. A new API allows developers to create plug-ins that implement both backup and restore functions for external backup software. This allows third-party solutions to be seamlessly integrated into the existing Proxmox backup stack and web interface. Fiona Ebner from Proxmox has shown what this can look like in a post for backups to directories and backups with Borg.

On the other hand, backup providers have the opportunity to connect their systems directly to Proxmox VE – including GUI integration and automated workflows. The plugins can use many internal functions, such as dirty bitmap tracking for fast incremental backups.

In addition, numerous improvements have been made to the existing backup mechanisms –, such as the elimination of race conditions in container backups, optimized change detection and more robust handling of fleecing backups, including automatic cleanup of orphaned images. The behavior of VMA template backups and file restores from the Proxmox Backup Server has also been standardized and stabilized, especially in connection with namespaces and encryption.

In the slipstream of VE 8.4: Proxmox Backup Server 3.4

Shortly after the release of Proxmox Virtual Environment 8.4, the developers also released Proxmox Backup Server 3.4. Both products are based on the same foundation.

In Backup Server 3.4, the garbage collection has been significantly optimized: it frees up storage space by removing data blocks (chunks) that are no longer required from the datastore. In the marking phase, in which it is determined which chunks are still being used by snapshots, a caching mechanism is now used. Although this increases the memory requirement, it reduces the runtime of the garbage collection and thus increases the efficiency of memory cleanup.

The control of sync jobs has also been enhanced. In addition to existing group filters, which can be used to specify which backup groups are synchronized, only encrypted or verified backup snapshots can now be transferred. In addition, the developers have created a statically linked build of the Proxmox Backup CLI client so that the client also runs on non-compatible Linux systems.

Mail gateway and Proxmox Datacenter Manager

The Proxmox Mail Getway was updated in February and is available as version 8.2. It is therefore quite up-to-date, which is why the next major update with the release of Debian 13 “Trixie” – can probably be expected in the summer –.

There is currently no news on the announced Proxmox Datacenter Manager, which is eagerly awaited by the community, but as company spokesperson Daniela Häsler from Vienna said, “… we are constantly working on it”. Interested parties can follow the current status and take a look at the alpha version.

Availability

All improvements and new features as well as possible problems when upgrading from Proxmox VE 8.3 to 8.4 are described in detail in the Proxmox roadmap. Proxmox VE 8.4, Proxmox Backup Server 3.4 and Proxmox Mail Gateway are now available for download as open-source software and can be used free of charge. Access to the enterprise repository is available from 115 euros (net) per year, professional support costs between 355 and 1060 euros (net) per year and CPU socket.

(emw)