Build a Nested KVM Hypervisor Lab with libvirt
You will turn one Linux server into a hypervisor that runs guest VMs, letting you build multi-node clusters on a single machine. By the end you will boot a nested guest and manage it entirely from the command line.
Learning Objectives
- Enable nested virtualization in the KVM kernel module.
- Provision a guest VM with libvirt and virt-install.
- Manage VM lifecycle with virsh.
- Time: ~3 hours · Difficulty: Advanced · Prereqs: a Linux host with CPU virtualization extensions and 8GB+ RAM.
Architecture Overview
Environment Setup
You will need: a Linux server with KVM (kvm-ok passes), libvirt, and virtinst installed. Note: kvm-ok ships in the cpu-checker package (sudo apt install -y cpu-checker), which the install command below does not include.
Before you begin: confirm CPU extensions with egrep -c '(vmx|svm)' /proc/cpuinfo (must be > 0).
Prerequisites
- A Linux host (or VM with nesting allowed) with virtualization extensions and internet access.
- Debian/Ubuntu for the apt commands shown (adapt package names for RHEL/Fedora).
- At least 8 GB RAM and 40 GB disk free for host plus guests.
- Root or sudo access; libvirt group membership for non-root management.
Step-by-Step Execution
These packages provide the KVM hypervisor, the libvirt management API, and the virt-install provisioning tool.
sudo apt install -y qemu-kvm libvirt-daemon-system virtinstExpected output:
Setting up qemu-kvm (1:8.2.2) ...
Setting up libvirt-daemon-system ...
Processing triggers for man-db ...What this teaches: qemu-kvm is the hypervisor, libvirt-daemon-system the management API, virtinst the provisioner; installing the trio gives both CLI and API control.
Nesting exposes virtualization extensions to guests so an L1 VM can itself run VMs.
echo 'options kvm_intel nested=1' | sudo tee /etc/modprobe.d/kvm.conf && sudo modprobe -r kvm_intel; sudo modprobe kvm_intel
Expected output:
(no output on success; verify with: cat /sys/module/kvm_intel/parameters/nested)
YWhat this teaches: options kvm_intel nested=1 persists the flag in modprobe.d so reboots keep nesting; reloading the module applies it without a reboot when no guests run. On AMD hosts the module is kvm_amd with the same nested=1 option. If modprobe -r fails because guests are running, skip the reload and reboot instead; the modprobe.d option takes effect at the next module load.
This downloads an Ubuntu cloud image, then builds a guest around it, scripted and repeatable.
sudo wget -q -O /var/lib/libvirt/images/node1.qcow2 https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.imgExpected output:
(no output on success; verify with: qemu-img info /var/lib/libvirt/images/node1.qcow2 | head -3)What this teaches: --import builds a guest around an existing disk image, so you must supply one first. Cloud images ship with cloud-init and grow-on-boot support, which is why they are the standard guest source for scripted labs.
sudo virt-install --name node1 --memory 2048 --vcpus 2 --disk /var/lib/libvirt/images/node1.qcow2,format=qcow2,bus=virtio --osinfo ubuntu22.04 --import --noautoconsole
Expected output:
Starting install...
Creating domain... | 0 B 00:00:00
Domain creation completed.What this teaches: --osinfo selects optimized virtual hardware for the guest OS (the older --osvariant flag still works as an alias); --import skips the installer and boots the existing image, making provisioning fully scripted. Warning: --import with --disk size=10 and no image path creates a blank disk the guest cannot boot from, which is the most common virt-install beginner error.
Expected output:
What this teaches: virsh list --all is the inventory view; from here console, snapshot-create-as, shutdown, and undefine cover the whole lifecycle.
How It Works
KVM turns the Linux kernel itself into a hypervisor via /dev/kvm, while QEMU provides the device emulation and libvirt offers a stable management API above both. This layering is why one virsh works across KVM, Xen, and even VMware: scripts target libvirt, not hypervisor quirks.
Nested virtualization exposes the host's VMX/SVM extensions to the L1 guest, letting it run KVM in turn. The use case is lab density: one physical host (or one large cloud VM) becomes a whole virtual datacenter for testing clustering, Proxmox, or Kubernetes topologies that need multiple nodes.
virt-install generates libvirt XML from flags, which matters for reproducibility: the same command provisions the same guest every time, and the XML can be versioned. virsh then manages the result through the full domain lifecycle, from define to undefine.
Progress So Far
Testing & Validation
cat /sys/module/kvm_intel/parameters/nested && virsh domstate node1 && virsh net-list --allYou should see Y for nesting, running for the guest, and the default NAT network active (it provides DHCP to guests; start it with virsh net-start default if it is inactive). If all three pass, your nested hypervisor lab works.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| No /dev/kvm device | Extensions disabled or the kvm module not loaded | Enable VT-x/AMD-V in firmware and modprobe kvm_intel (or kvm_amd) |
| Default network has no DHCP | libvirt's default NAT network not started | Run virsh net-start default and virsh net-autostart default |
| Guest console shows nothing | No console device configured on the domain | Add --graphics none --console pty,target_type=serial for text consoles |
| Permission denied on virsh | User outside the libvirt group | usermod -aG libvirt $USER, then log out and back in |
| virt-install hangs at install | Waiting on interactive installer input | Use --noautoconsole with cloud images, or provide automated install media |
| Nested L2 extremely slow | Double virtualization overhead | Accept for labs; reduce vCPUs and use host-passthrough CPU model |
Troubleshooting
- nested reads N: reload the module after adding the modprobe option; persist via
/etc/modprobe.d. - Guest has no virtualization: set the L1 CPU mode to host-passthrough.
- libvirtd not running:
sudo systemctl enable --now libvirtd.
Extension Ideas
- Boot two guests and build the Kubernetes Basics lab across them.
- Compare with the Proxmox Homelab approach.
- Automate guest creation with cloud-init user-data.
Key Results
- Enabled nested KVM and booted a guest inside a VM.
- Provisioned guests reproducibly with one virt-install command.
- Built multi-node topologies on a single physical host.
- Managed full VM lifecycle through virsh.
Key Takeaways
- KVM plus libvirt plus virt-install is the complete open-source stack.
- Persist nested=1 in modprobe.d for reboot survival.
- virt-install flags generate reproducible libvirt XML.
- virsh covers the entire guest lifecycle from the CLI.
- Start libvirt's default network or guests get no DHCP.
Practice Challenges
Challenge 1: Cluster in a box
Goal: provision three nested guests, give them static IPs, and get passwordless SSH between all pairs.
Hint: cloud-init NoCloud seeds set users and keys.
Challenge 2: Snapshot policy
Goal: script nightly virsh snapshots of a guest with a 7-day rotation, logging each run.
Hint: snapshot-create-as with dated names plus a prune loop.