I built flexible storage with LVM, layering physical volumes into a volume group and carving logical volumes that resize online. Growing a mounted filesystem with lvextend plus an online resize delivered more space with zero downtime.

Objective & Context

LVM decouples filesystems from physical disks, enabling live growth and snapshots. This lab builds the PV/VG/LV stack, extends a volume online, and takes a snapshot for consistent backups, the storage flexibility behind the server and backup labs.

Learning Objectives

  • Initialize physical volumes and pool them into a volume group.
  • Carve logical volumes out of a volume group and format them.
  • Extend a logical volume and its filesystem while mounted.
  • Create LVM snapshots for point-in-time consistent backups.
  • Explain the PV/VG/LV layering and when LVM beats plain partitions.

Environment & Prerequisites

Prerequisites

  • Linux with LVM2 installed: sudo apt install -y lvm2 or sudo dnf install -y lvm2.
  • Two spare block devices or loop files (never practice on disks with data).
  • An ext4 or xfs filesystem on the LV (resize tools differ slightly).
  • Root access; every storage operation requires elevation.

Step-by-Step Execution

1. Build the LVM stack [ROOT REQUIRED]

pvcreate /dev/sdb /dev/sdc && vgcreate data /dev/sdb /dev/sdc && lvcreate -L 10G -n vol data

Expected output:

  Physical volume "/dev/sdb" successfully created.
  Physical volume "/dev/sdc" successfully created.
  Volume group "data" successfully created
  Logical volume "vol" created.
mkfs.ext4 -L data-vol /dev/data/vol && mkdir -p /mnt/data && mount /dev/data/vol /mnt/data && df -h /mnt/data

Expected output:

Filesystem      Size  Used Avail Use% Mounted on
/dev/mapper/data-vol  9.8G   24K  9.3G   1% /mnt/data

What this teaches: pvcreate labels disks for LVM; vgcreate pools their extents into one allocatable space; lvcreate carves a 10G virtual partition from the pool. Three layers, each with one job. The LV is a raw block device until mkfs builds a filesystem on it and mount attaches it, which is why both steps are needed before any resize can be demonstrated. pvcreate destroys existing data on the target devices: double-check the device names with lsblk first, and on a machine with no spare disks, practice on loop files (fallocate -l 5G /tmp/disk1.img, then losetup -fP) instead.

2. Grow the volume and filesystem online

lvextend -L +5G /dev/data/vol && resize2fs /dev/data/vol

Expected output:

  Size of logical volume data/vol changed from 10.00 GiB (2560 extents) to 15.00 GiB (3840 extents).
  Logical volume data/vol successfully resized.
resize2fs 1.47.0 (5-Feb-2023)
Resizing the filesystem on /dev/mapper/data-vol to 3932160 (4k) blocks.
The filesystem on /dev/mapper/data-vol is now 3932160 (4k) blocks long.

What this teaches: lvextend grows the block device; resize2fs (or xfs_growfs /mnt/data for XFS) grows the filesystem into it. Both run online: no unmount, no downtime, which plain partitions cannot do. Shortcut: lvextend -r -L +5G grows the filesystem in the same command and handles ext4 and XFS automatically.

3. Snapshot for a consistent backup

lvcreate -s -L 2G -n vol-snap /dev/data/vol

Expected output:

  Logical volume "vol-snap" created.

What this teaches: A snapshot is a copy-on-write clone frozen at creation time: mount it (mount -o ro /dev/data/vol-snap /mnt/snap) and back up a consistent image while the live volume keeps changing underneath. The 2G size only needs to cover blocks that change during the backup window, not the full 15G, but if the origin diverges more than 2G the snapshot invalidates.

How It Works

LVM inserts a virtualization layer between disks and filesystems. Physical volumes contribute extents (typically 4 MB chunks) to a volume group, which is a pool of allocatable space; logical volumes are carved from the pool and presented as block devices. Because the mapping is logical, an LV can span disks, grow, shrink, or move while mounted.

Online resize works in two phases because the block device and the filesystem are separate layers. lvextend tells the device-mapper the LV is bigger; the filesystem then needs its own grow step (resize2fs for ext4, xfs_growfs for XFS, and XFS only grows). Skipping the second phase is the classic 'I extended but df shows nothing' bug.

Snapshots exploit copy-on-write: at creation the snapshot shares all extents with the origin, and only changed blocks are copied aside afterwards. This gives a frozen, mountable image for backups, but the snapshot fills as the origin diverges, so size it for the expected change rate during the backup window.

Validation & Testing

Confirm df -h shows the larger size while the filesystem stayed mounted, and that the snapshot is usable for backup. Pass criteria: online growth with zero downtime and a consistent snapshot for point-in-time backup.

Troubleshooting

SymptomLikely causeFix
df shows no new space after lvextendThe filesystem was not grown, only the block deviceRun resize2fs (ext4) or xfs_growfs (XFS) on the mounted LV
Insufficient free extents errorThe VG has no unallocated space leftAdd another PV with vgextend, or extend by a smaller amount
Snapshot ran out of space mid-backupOrigin diverged more than the snapshot size allowedSize snapshots for the change rate, monitor with lvs, and back up promptly
Cannot reduce an XFS volumeXFS does not support shrinkingPlan XFS sizes generously, or use ext4 where shrink is needed
VG not found after rebootLVM metadata scan missed the devices (rare on modern initramfs)Run vgchange -ay and check device filters in lvm.conf
Removing an LV says it is in useIt is mounted or a snapshot depends on itUnmount first, merge or remove snapshots, then lvremove
Advanced: Troubleshooting
  • Resize didn't apply: extend the LV first, then grow the filesystem (resize2fs/xfs_growfs).
  • Snapshot fills up: size snapshots for expected change rate or they invalidate.
  • VG out of space: add a PV with pvcreate and vgextend before lvextend.

Key Results

  • Grew a mounted filesystem online with zero downtime.
  • Pooled multiple disks into one flexible volume group.
  • Captured consistent snapshots for point-in-time backups.
  • Decoupled filesystem size from physical disk boundaries.

Key Takeaways

  • PVs feed extents into a VG; LVs are carved from the VG pool.
  • Resize is two steps: lvextend for the device, filesystem grow second.
  • Snapshots give frozen, mountable images for consistent backups.
  • Size snapshots for the change rate during the backup window.
  • XFS grows only; plan accordingly.

Practice Challenges

Challenge 1: Online growth drill

Goal: fill an LV to 90%, extend it by 5G online without unmounting, and prove with df that no downtime occurred.

Hint: watch lvs and df -h side by side.

Challenge 2: Snapshot backup

Goal: snapshot a live LV, mount the snapshot read-only, tar it to an archive, then remove the snapshot.

Hint: mount -o ro the snapshot device under /dev/data/.