Nested Virtual Machines with KVM, libvirt, and virsh
I enabled nested virtualization on KVM and provisioned guest VMs inside a VM with libvirt, virt-install, and virsh. Nesting let me build multi-node clustering and hypervisor labs on a single physical host with no additional hardware.
Objective & Context
Nested KVM exposes the CPU virtualization extensions to a guest so it can itself run VMs. This lab turns on nesting, scripts guest creation with virt-install, and manages lifecycle with virsh, the technique behind the Proxmox and Kubernetes cluster labs.
Learning Objectives
- Verify CPU virtualization extensions and enable KVM nesting.
- Persist the nested=1 module option across reboots.
- Provision inner guests non-interactively with virt-install.
- Manage guest lifecycle (start, shutdown, snapshot) with virsh.
- Explain the performance and use-case trade-offs of nesting.
Environment & Prerequisites
Prerequisites
- A Linux host with KVM, libvirt, and CPU virtualization extensions (or a cloud VM with nesting allowed).
- Sufficient RAM and CPU for nested guests (2 vCPU and 2 GB per inner guest minimum).
- A cloud image or ISO for the inner guest.
- Root or sudo access and membership in the libvirt group for management.
Step-by-Step Execution
The outputs on this page come from a re-run on 14 November 2026 in an Ubuntu 24.04 container (arm64). Sample files, users, and hosts were created for the run, so names, numbers, and timestamps reflect that run. Steps that could not be re-run there are marked as example output, with the reason.
1. Enable nested virtualization [ROOT REQUIRED]
sudo modprobe -r kvm_intel; sudo modprobe kvm_intel nested=1; cat /sys/module/kvm_intel/parameters/nestedExample output (not re-run: nested KVM needs an Intel host with hardware virtualization):
Y
What this teaches: kvm_intel nested=1 exposes VMX to guests so the L1 guest can run its own hypervisor; reading the parameter back confirms the module accepted it (Y means enabled, N means disabled). The kernel has enabled nesting by default since v4.20, so on a modern host this often already reads Y and the reload is a confirmation. To persist across reboots regardless of defaults, write options kvm_intel nested=1 to /etc/modprobe.d/kvm-nested.conf with echo "options kvm_intel nested=1" | sudo tee /etc/modprobe.d/kvm-nested.conf; on AMD hosts the module is kvm_amd. The reload fails with "module in use" while guests run, so shut them down first.
2. Provision an inner guest with virt-install
sudo virt-install --name inner1 --memory 2048 --vcpus 2 --disk size=10 --osinfo debian12 --cdrom /var/lib/libvirt/images/debian-12-amd64-netinst.iso --noautoconsoleExample output (not re-run: nested KVM needs an Intel host with hardware virtualization):
Starting install...
Creating domain... | 10.0 GB
Domain creation completed.
What this teaches: virt-install turns guest creation into one scripted command: name, memory, vCPUs, disk, and OS profile (--osinfo) pick the right virtual hardware automatically. Note the install media choice: --cdrom boots the installer for a fresh OS install, while --import is only valid when --disk points at an already installed disk image (it skips the installer and builds the guest around it). Using --import together with --disk size=10 and no image path fails, so pick the flag that matches your media: an ISO gets --cdrom, a downloaded cloud image gets --disk /path/to/image.qcow2 --import. sudo is needed for the default qemu:///system connection; members of the libvirt group can drop it.
3. Manage with virsh
virsh list --all && virsh start inner1Example output (not re-run: nested KVM needs an Intel host with hardware virtualization):
Id Name State
--------------------------
3 inner1 running
Example output (not re-run: nested KVM needs an Intel host with hardware virtualization):
Domain 'inner1' started
What this teaches: virsh list --all shows every defined guest and state; virsh start boots by name. virsh is the CLI for the full libvirt API: console, snapshot, migrate, destroy.
How It Works
Nested virtualization works by passing the CPU's virtualization extensions through to the guest. Normally the hypervisor hides VMX/SVM from guests; with nesting enabled, the L1 guest sees the extensions and can run KVM itself, creating L2 guests. Each level adds trap-and-emulate overhead, so nested guests run slower, but for lab and CI purposes the convenience outweighs the cost.
libvirt is the management layer that turns hypervisor APIs into a stable interface: virsh commands, XML domain definitions, and storage/network pools. virt-install is a thin provisioner on top that generates the XML from flags, which is why scripted lab environments prefer it over clicking through virt-manager.
The canonical use case is hypervisor development and training: the Proxmox and Kubernetes labs in the work section rely on nested guests to simulate multi-node clusters on one physical host. Understanding the nesting toggle is what makes those environments reproducible anywhere.
Validation & Testing
Confirm nested reads Y, then boot an inner guest and verify it sees virtualization extensions (egrep -c '(vmx|svm)' /proc/cpuinfo > 0). Pass criteria: nesting enabled, inner guest boots, and it can itself run a container or VM.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| modprobe: module in use | Guests are running, so kvm_intel cannot reload | Shut down all guests first, or set the option via /etc/modprobe.d and reboot |
| Inner guest reports KVM not available | Nesting did not apply to the L1 guest | Confirm Y in /sys/module/kvm_intel/parameters/nested on the host |
| virt-install: --osinfo unknown | osinfo-db is outdated or missing | Update osinfo-db or use --os-variant with a close match |
| virsh: permission denied | User is not in the libvirt group | Add with usermod -aG libvirt $USER and log back in |
| Nested guest is painfully slow | Expected: double trap-and-emulate overhead | Reduce L2 vCPUs, enable host CPU passthrough, accept it for labs only |
| Cloud provider blocks nesting | The provider hides virtualization extensions | Use a provider/instance type with nested virtualization support |
Advanced: Troubleshooting
- nested = N: the module must be reloaded with
nested=1; persist via modprobe.d. - Inner guest no VT: set the L1 VM CPU mode to host-passthrough.
- Poor performance: nesting adds overhead; allocate generous host resources.
Key Results
- Enabled nested KVM and booted guests inside a VM.
- Built multi-node lab topologies on a single physical host.
- Scripted reproducible guest creation with virt-install.
- Managed full VM lifecycle through virsh.
Key Takeaways
- nested=1 exposes VMX/SVM to guests, enabling L2 VMs.
- Persist module options in /etc/modprobe.d, not just modprobe.
- virt-install scripts guest creation; virsh manages lifecycle.
- Nesting costs performance; it is a lab tool, not a production pattern.
- libvirt group membership controls who can manage guests.
Practice Challenges
Challenge 1: Two-level lab
Goal: on the L1 guest, install KVM and boot an L2 guest, then prove nesting with lscpu inside L2.
Hint: repeat the host steps inside L1 after confirming the flag.
Challenge 2: Snapshot round-trip
Goal: snapshot inner1, break its network config, then revert and confirm connectivity returns.
Hint: virsh snapshot-create-as and snapshot-revert.
Frequently Asked Questions
How do I fix 'modprobe: module in use'?
Likely cause: Guests are running, so kvm_intel cannot reload. Fix: Shut down all guests first, or set the option via /etc/modprobe.d and reboot.
How do I fix 'Inner guest reports KVM not available'?
Likely cause: Nesting did not apply to the L1 guest. Fix: Confirm Y in /sys/module/kvm_intel/parameters/nested on the host.
How do I fix 'virt-install: --osinfo unknown'?
Likely cause: osinfo-db is outdated or missing. Fix: Update osinfo-db or use --os-variant with a close match.
How do I fix 'virsh: permission denied'?
Likely cause: User is not in the libvirt group. Fix: Add with usermod -aG libvirt $USER and log back in.
How do I fix 'Nested guest is painfully slow'?
Likely cause: Expected: double trap-and-emulate overhead. Fix: Reduce L2 vCPUs, enable host CPU passthrough, accept it for labs only.
Cite This Page
Citation formats
Plain text
Taki Sadik, "Nested Virtual Machines with KVM, libvirt, and virsh", TYFSADIK, https://tyfsadik.org/blog/labs/linux-fundamentals/nested-vms-kvm (accessed 2026-10-22).
BibTeX
@misc{sadik-nested-vms-kvm,
author = {Sadik, MD. Taki Yasir Faraji},
title = {Nested Virtual Machines with KVM, libvirt, and virsh},
year = {2026},
url = {https://tyfsadik.org/blog/labs/linux-fundamentals/nested-vms-kvm},
urldate = {2026-10-22}
}
APA
Sadik, M. T. Y. F. (2026). Nested Virtual Machines with KVM, libvirt, and virsh. TYFSADIK. https://tyfsadik.org/blog/labs/linux-fundamentals/nested-vms-kvm