Home › Projects › Infrastructure › Proxmox Homelab

Proxmox Homelab

Full virtualization infrastructure for testing and development

2025 Infrastructure GitHub ↗

Overview

This project is a bare-metal Proxmox VE hypervisor deployment that serves as the foundation for every other self-hosted service in the stack. Proxmox VE combines KVM full virtualization and LXC container technology on a single platform managed through a polished web UI. The server runs on commodity x86-64 hardware with a ZFS mirror storage pool providing data integrity, transparent checksumming, and instant copy-on-write snapshots.

The network topology uses Linux bridge interfaces with VLAN-aware configuration, allowing VMs and containers to be isolated into separate broadcast domains without purchasing managed switches. Automated backups run nightly via vzdump with zstd compression, and cloud-init VM templates enable rapid provisioning of new guests in under a minute. Ansible playbooks automate post-provisioning hardening so every new VM is immediately ready to receive a service. This single server currently hosts the DNS resolver, email server, Nextcloud instance, Immich photo server, and the search engine.

Architecture

graph TD BM["Bare Metal Host\nx86-64 Hardware"] BM --> PVE["Proxmox VE 8\nHypervisor + Web UI"] PVE --> KVM["KVM Virtual Machines"] PVE --> LXC["LXC Containers"] PVE --> ZFS["ZFS Storage Pool\n(mirror: sdb + sdc)"] PVE --> NET["Network Bridges\nvmbr0 / vmbr1"] KVM --> VM1["Debian 12 VM\ndev / staging"] KVM --> VM2["Ubuntu 22.04 VM\ntesting"] KVM --> VM3["Windows 11 VM\ncompatibility testing"] LXC --> CT1["DNS Container\nPi-hole + Unbound"] LXC --> CT2["Mail Container\nPostfix + Dovecot"] LXC --> CT3["App Containers\nNextcloud / Immich"] LXC --> CT4["Proxy Container\nNginx"] ZFS --> DS1[("local-zfs\nVM disks")] ZFS --> DS2[("tank/backups\nvzdump archives")] NET --> VLAN10["VLAN 10\nManagement"] NET --> VLAN20["VLAN 20\nServices"] style BM fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style PVE fill:#1a1a2e,stroke:#00ff88,color:#e0e0e0 style KVM fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style LXC fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style ZFS fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style NET fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style VM1 fill:#181818,stroke:#1e1e1e,color:#888 style VM2 fill:#181818,stroke:#1e1e1e,color:#888 style VM3 fill:#181818,stroke:#1e1e1e,color:#888 style CT1 fill:#181818,stroke:#1e1e1e,color:#888 style CT2 fill:#181818,stroke:#1e1e1e,color:#888 style CT3 fill:#181818,stroke:#1e1e1e,color:#888 style CT4 fill:#181818,stroke:#1e1e1e,color:#888 style DS1 fill:#181818,stroke:#1e1e1e,color:#888 style DS2 fill:#181818,stroke:#1e1e1e,color:#888 style VLAN10 fill:#181818,stroke:#1e1e1e,color:#888 style VLAN20 fill:#181818,stroke:#1e1e1e,color:#888

Tech Stack

  • Proxmox VE 8 — open-source hypervisor platform built on Debian
  • KVM / QEMU — full hardware virtualization for Windows and mixed-OS VMs
  • LXC — lightweight Linux containers sharing the host kernel
  • ZFS — copy-on-write filesystem with mirror RAID, checksums, and snapshots
  • Linux Bridges / VLAN — software-defined networking with VLAN-aware bridges
  • Ansible — idempotent playbooks for guest provisioning and hardening
  • vzdump — Proxmox native backup tool with zstd compression and rotation

Build Process

1

Install Proxmox VE on Bare Metal

The Proxmox VE ISO is written to a USB drive and booted on the target machine. During installation, ZFS RAID-1 is selected so the OS itself is mirrored across two drives. The management IP, gateway, and hostname are configured at this stage. After first boot, the enterprise apt repository is replaced with the no-subscription community repository to avoid paid-tier update errors. 

# Write ISO to USB from another machine
dd if=proxmox-ve_8.2-1.iso of=/dev/sdX bs=4M status=progress conv=fdatasync

# After first boot: switch to community repository
echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \
  > /etc/apt/sources.list.d/pve-community.list

# Disable enterprise repo
rm /etc/apt/sources.list.d/pve-enterprise.list

apt update && apt full-upgrade -y
reboot
2

Configure ZFS Storage Pool

Additional data drives are formed into a ZFS mirror pool named tank for redundant VM and backup storage. ZFS ARC memory usage is capped to prevent it from consuming all available RAM and causing OOM kills in guest VMs. Two datasets are created with separate mount points. 

# Create ZFS mirror pool across two data drives
zpool create -f tank mirror /dev/sdb /dev/sdc

# Create datasets
zfs create tank/vmdisks
zfs create tank/backups

# Cap ARC at 4 GB (tune to ~25% of total RAM)
echo "options zfs zfs_arc_max=4294967296" > /etc/modprobe.d/zfs.conf
update-initramfs -u

# Verify pool health
zpool status tank
zfs list -r tank
3

Set Up Network Bridges and VLANs

vmbr0 bridges the physical NIC for internet-routed traffic. vmbr1 is a VLAN-aware internal bridge for inter-VM communication, enabling each VM port to carry a VLAN tag set from the Proxmox web UI without requiring external managed switch configuration. 

# /etc/network/interfaces
auto lo
iface lo inet loopback

auto enp3s0
iface enp3s0 inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.1.10/24
    gateway 192.168.1.1
    bridge-ports enp3s0
    bridge-stp off
    bridge-fd 0

auto vmbr1
iface vmbr1 inet manual
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes
    bridge-vids 2-4094

# Apply without reboot
ifreload -a
4

Create VM and LXC Templates

A Debian 12 cloud-init VM template is built from the official generic cloud image. This template includes cloud-init drive attachment so that SSH keys and hostname are injected on first boot. LXC container templates are downloaded from the Proxmox template repository for fast container provisioning. 

# Download Debian 12 cloud image
wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-genericcloud-amd64.qcow2

# Create template VM (ID 9000)
qm create 9000 --name debian-12-tmpl --memory 2048 --cores 2 \
  --net0 virtio,bridge=vmbr0 --scsihw virtio-scsi-pci
qm importdisk 9000 debian-12-genericcloud-amd64.qcow2 local-zfs
qm set 9000 --scsi0 local-zfs:vm-9000-disk-0 --boot c --bootdisk scsi0
qm set 9000 --ide2 local-zfs:cloudinit --serial0 socket --vga serial0
qm set 9000 --sshkey ~/.ssh/id_ed25519.pub --ciuser admin
qm template 9000

# Download and list LXC templates
pveam update
pveam download local debian-12-standard_12.7-1_amd64.tar.zst
5

Configure Automated Backups

Backup jobs are configured via /etc/pve/jobs.cfg and exposed in the Proxmox web UI under Datacenter → Backup. Backups run nightly at 02:00 using zstd compression in snapshot mode (no downtime). The maxfiles value of 7 enforces a weekly rolling retention window. 

# /etc/pve/jobs.cfg (managed by Proxmox)
vzdump: job-nightly
    schedule 0 2 * * *
    storage tank/backups
    compress zstd
    mode snapshot
    maxfiles 7
    all 1

# Manual backup of a specific guest
vzdump 101 --storage tank/backups --compress zstd --mode snapshot

# List existing backup archives
pvesm list tank/backups

# Restore from backup
qmrestore /mnt/pve/tank/dump/vzdump-qemu-101-*.vma.zst 200 --storage local-zfs
6

Deploy Services as LXC Containers

Services are provisioned as unprivileged LXC containers cloned from the Debian template. Each container receives a static IP on vmbr1 with an appropriate VLAN tag. An Ansible playbook runs immediately after creation to install Docker, configure the firewall, and bring up the service compose stack. 

# Create unprivileged LXC container for DNS service
pct create 101 local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \
    --hostname dns-server \
    --cores 1 --memory 512 --swap 512 \
    --net0 name=eth0,bridge=vmbr1,ip=192.168.20.2/24,gw=192.168.20.1,tag=20 \
    --storage local-zfs --rootfs local-zfs:8 \
    --unprivileged 1 --features keyctl=1,nesting=1 \
    --start 1

# Run Ansible provisioning playbook
ansible-playbook -i inventory/homelab.yml playbooks/provision-lxc.yml \
    --limit dns-server --tags docker,hardening

Data Flow

flowchart TD A["Select Template\nVM or LXC"] --> B["Clone / Create Guest\nqm clone or pct create"] B --> C["Set Resources\nCPU, RAM, disk size"] C --> D["Assign VLAN Tag\nvmbr1 + tag ID"] D --> E["First Boot\npct start or qm start"] E --> F["Cloud-init Runs\nSSH key, hostname inject"] F --> G["Ansible Playbook\nhardening + Docker install"] G --> H["Deploy Service Stack\ndocker compose up -d"] H --> I["Nightly Backup\nvzdump to tank/backups"] I --> J["ZFS Snapshot\nauto-snapshot or manual"] style A fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style B fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style C fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style D fill:#181818,stroke:#1e1e1e,color:#888 style E fill:#1a1a2e,stroke:#00ff88,color:#e0e0e0 style F fill:#181818,stroke:#1e1e1e,color:#888 style G fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style H fill:#1a1a2e,stroke:#00ff88,color:#e0e0e0 style I fill:#181818,stroke:#1e1e1e,color:#888 style J fill:#181818,stroke:#1e1e1e,color:#888

Challenges & Solutions

  • ZFS ARC consuming all available RAM: By default ZFS ARC expands to fill roughly half of system RAM and does not release it under normal memory pressure, causing OOM kills in guest VMs. Resolved by setting zfs_arc_max=4294967296 in /etc/modprobe.d/zfs.conf and regenerating the initramfs with update-initramfs -u.
  • VLAN tagging not working on guest VMs: VMs assigned VLAN tags still communicated across all VLANs. The issue was that vmbr1 lacked the bridge-vlan-aware yes option. After adding it and running ifreload -a, VLAN isolation functioned correctly.
  • LXC containers unable to run Docker: Unprivileged LXC containers block nested container runtimes due to missing kernel capabilities. Resolved by setting features: keyctl=1,nesting=1 in the container configuration and selecting the overlay storage driver.
  • Cloud-init not injecting SSH keys: The cloud-init drive was attached but user data was not set before templating. Fixed with qm set 9000 --sshkey ~/.ssh/id_ed25519.pub --ciuser admin before converting the VM to a template.

What I Learned

  • KVM vs LXC trade-offs: full isolation overhead vs shared-kernel efficiency
  • ZFS architecture: datasets, datasets inheritance, ARC tuning, and scrub schedules
  • Linux bridge networking, VLAN-aware bridge configuration, and traffic isolation
  • Cloud-init for automated VM customization at first boot
  • Proxmox backup strategies: snapshot vs suspend vs stop modes, retention policies
  • Ansible inventory management and idempotent playbook design for homelab automation
Proxmox KVM LXC ZFS Virtualization Homelab