ZFS Pool Management Guide (Debian/Ubuntu)

Introduction

ZFS (Zettabyte File System) is an advanced filesystem and volume manager for large storage pools, offering snapshots, compression, and redundancy (e.g., mirror, RAID-Z). This guide shows how to initialize disks with GPT, create a ZFS pool, use stable device identifiers (/dev/disk/by-id/ or /dev/disk/by-partuuid/), support various RAID types (mirror, RAID-Z1, RAID-Z2), and mount datasets for normal user access on Debian/Ubuntu.

⚡ Quick Start

What is a ZFS Pool?

A ZFS pool is a collection of virtual devices (vdevs) forming the foundation for ZFS datasets (filesystems) and zvols (block devices). Pools support:

• Redundancy: Mirror (RAID-1), RAID-Z1/2/3 (like RAID-5/6/7).
• Expansion: Add vdevs to increase capacity (cannot remove easily).
• Health Monitoring: Scrubbing and checksums for data integrity.
• Snapshots/Clones: Point-in-time copies without full duplication.

Pools are created with zpool create and managed via zpool commands. Data is stored in datasets (e.g., tank/home).

🔑 Understanding UUIDs in ZFS

Pool UUID vs Device Identifiers

Pool UUID

• Example: 150809157285762621
• Shown as UUID="..." with TYPE="zfs_member" in blkid output
• Shared by ALL disks/partitions that are members of the same pool
• Not used directly in zpool commands

Device Identifiers (for referencing individual disks/partitions)

/dev/disk/by-id/ - Persistent identifier based on disk serial number

• ✅ Best for whole disks
• Example: ata-Samsung_SSD_850_S21NX0AG123456
• Survives disk reordering and system reboots

/dev/disk/by-partuuid/ - GPT partition UUID

• ✅ Best for manual partitions
• Example: ee2507fe-0b11-ad4c-b1c5-87e36055410e
• Each partition has its own unique PARTUUID

/dev/vdb, /dev/sda - Kernel device names

• ⚠️ Unreliable - can change on reboot or disk reordering
• Use only for initial pool creation, then switch to persistent identifiers

Decision Matrix: Which Identifier to Use?

Prerequisites

Installation

sudo apt update
sudo apt install zfsutils-linux parted
sudo modprobe zfs

Verify Installation

zpool status  # No pools initially
zfs list      # Empty initially

Identifying Available Disks

1. List All Block Devices

# Show all disks and their mount status
lsblk

Example output:

NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0   20G  0 disk 
└─sda1   8:1    0   20G  0 part /
sr0     11:0    1 1024M  0 rom  
vdb    252:16   0   10G  0 disk 
vdc    252:32   0   10G  0 disk 

In this example, vdb and vdc are available disks without partitions.

2. Check Disk Identifiers

# Try to list persistent IDs
ls -l /dev/disk/by-id/ | grep -v part

What you might see:

# Only DVD/CD-ROM visible, no data disks
lrwxrwxrwx 1 root root 9 ott  4 11:50 ata-QEMU_DVD-ROM_QM00001 -> ../../sr0

3. Verify Disks Are Empty

# Check if disks have any filesystem or partition table
sudo blkid /dev/vdb /dev/vdc

# If empty, no output or "no such device"

# Alternative check with wipefs
sudo wipefs /dev/vdb /dev/vdc

🚀 Recommended Approach: Whole Disks

Method A: With /dev/disk/by-id/ (Physical Disks)

If your disks appear in /dev/disk/by-id/ (typical for physical SATA/SAS disks):

# List available disk IDs
ls -l /dev/disk/by-id/ | grep -v part

# Example output for physical disks:
# ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567 -> ../../sdb
# ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7654321 -> ../../sdc

Use these IDs directly in pool creation.

Method B: Without /dev/disk/by-id/ (Virtual Disks)

Option 1: Create pool with device names (Recommended for VMs)

ZFS automatically creates a GPT partition table when you use a whole disk.

# Create pool with device names
sudo zpool create tank mirror /dev/vdb /dev/vdc

# After creation, check what ZFS created:
sudo blkid | grep zfs_member
ls -l /dev/disk/by-id/ | grep -v dvd

# Check for virtio identifiers
ls -l /dev/disk/by-id/ | grep virtio

# If available, use them:
# virtio-xxxxx -> ../../vdb
sudo zpool create tank mirror \
  /dev/disk/by-id/virtio-xxxxx \
  /dev/disk/by-id/virtio-yyyyy

Creating the Pool (Whole Disks)

ZFS will automatically create a GPT partition table and use the entire disk.

With Physical Disks (using by-id)

Mirror (RAID-1, 2 disks) - Recommended for most users

sudo zpool create tank mirror \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7654321

sudo zpool create tank raidz \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7654321 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K9876543

sudo zpool create tank raidz2 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7654321 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K9876543 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1111111

With Virtual Disks (using device names)

Mirror (RAID-1, 2 disks) - Most common for VMs

# ZFS creates GPT automatically
sudo zpool create tank mirror /dev/vdb /dev/vdc

sudo zpool create tank raidz /dev/vdb /dev/vdc /dev/vdd

sudo zpool create tank raidz2 /dev/vdb /dev/vdc /dev/vdd /dev/vde

Verify Pool Status

zpool status tank

Example output (physical disks with by-id):

  pool: tank
 state: ONLINE
config:
        NAME                                            STATE     READ WRITE CKSUM
        tank                                            ONLINE       0     0     0
          mirror-0                                      ONLINE       0     0     0
            ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567    ONLINE       0     0     0
            ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7654321    ONLINE       0     0     0

Example output (virtual disks):

  pool: tank
 state: ONLINE
config:
        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            vdb     ONLINE       0     0     0
            vdc     ONLINE       0     0     0

Understanding What ZFS Created

After creating a pool with whole disks, check what ZFS automatically created:

# Show partition table ZFS created
sudo blkid | grep zfs_member

Example output (virtual disks):

/dev/vdb: UUID="150809157285762621" UUID_SUB="4234567890123456789" TYPE="zfs_member" PTTYPE="gpt"
/dev/vdc: UUID="150809157285762621" UUID_SUB="9876543210987654321" TYPE="zfs_member" PTTYPE="gpt"

Or with partitions:

/dev/vdb1: UUID="150809157285762621" TYPE="zfs_member" PARTUUID="ee2507fe-0b11-ad4c-b1c5-87e36055410e"
/dev/vdc1: UUID="150809157285762621" TYPE="zfs_member" PARTUUID="c074a830-4f67-a24d-b028-7cefbe64a690"

# Check partition layout
lsblk

What ZFS created:

• Partition 1 (vdb1, vdc1): Main ZFS partition with your data
• Partition 9 (vdb9, vdc9): Small 8MB partition for ZFS reserved area
• GPT (PTTYPE="gpt"): GUID Partition Table
• Pool UUID: Same for all pool members (150809157285762621)
• PARTUUID: Unique for each partition (if created)

Alternative Approach: Manual Partitions with by-partuuid

# Initialize disk with GPT
sudo parted /dev/vdb mklabel gpt
sudo parted /dev/vdb mkpart primary 0% 100%

sudo parted /dev/vdc mklabel gpt
sudo parted /dev/vdc mkpart primary 0% 100%

# List partition UUIDs
ls -l /dev/disk/by-partuuid/

# Or use blkid
sudo blkid /dev/vdb1 /dev/vdc1

/dev/vdb1: PARTUUID="ee2507fe-0b11-ad4c-b1c5-87e36055410e"
/dev/vdc1: PARTUUID="c074a830-4f67-a24d-b028-7cefbe64a690"

sudo zpool create tank mirror \
  /dev/disk/by-partuuid/ee2507fe-0b11-ad4c-b1c5-87e36055410e \
  /dev/disk/by-partuuid/c074a830-4f67-a24d-b028-7cefbe64a690

# Create with device names
sudo zpool create -f tank mirror /dev/vdb1 /dev/vdc1

# Export and re-import with PARTUUIDs
sudo zpool export tank
sudo zpool import -d /dev/disk/by-partuuid tank

zpool status tank

  pool: tank
 state: ONLINE
config:
        NAME                                     STATE     READ WRITE CKSUM
        tank                                     ONLINE       0     0     0
          mirror-0                               ONLINE       0     0     0
            ee2507fe-0b11-ad4c-b1c5-87e36055410e ONLINE       0     0     0
            c074a830-4f67-a24d-b028-7cefbe64a690 ONLINE       0     0     0

💾 Creating and Managing Datasets

Create and Mount Dataset

# Create dataset
sudo zfs create tank/data

# Set mountpoint
sudo zfs set mountpoint=/mnt/tank tank/data

# Enable compression (recommended)
sudo zfs set compression=lz4 tank/data

Set Permissions for Normal User

# Change ownership to your user
sudo chown manzolo:manzolo /mnt/tank
sudo chmod 775 /mnt/tank

Write Data as Normal User

# Now you can write without sudo
echo "Test data" > /mnt/tank/test.txt
cat /mnt/tank/test.txt

Verify Dataset

zfs list
df -h /mnt/tank

🔧 Pool Modification Operations

Add Devices to Expand Pool

sudo zpool add tank mirror \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K3333333 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K4444444

sudo zpool add tank raidz \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5555555 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6666666 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7777777

# Create partitions
sudo parted /dev/vde mklabel gpt
sudo parted /dev/vde mkpart primary 0% 100%
sudo parted /dev/vdf mklabel gpt
sudo parted /dev/vdf mkpart primary 0% 100%

# Add to pool using PARTUUIDs
sudo zpool add tank mirror \
  /dev/disk/by-partuuid/[partuuid-of-vde1] \
  /dev/disk/by-partuuid/[partuuid-of-vdf1]

Replace a Failed Disk

For Pools Created with Whole Disks (by-id)

1. Check pool status to identify failed disk:

zpool status tank

Example output showing failure:

  pool: tank
 state: DEGRADED
config:
        NAME                                            STATE     READ WRITE CKSUM
        tank                                            DEGRADED     0     0     0
          mirror-0                                      DEGRADED     0     0     0
            ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567    UNAVAIL      0     0     0
            ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7654321    ONLINE       0     0     0

2. Replace the failed disk:

sudo zpool replace tank \
  ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K9999999

3. Monitor resilvering:

zpool status tank

zpool status tank

sudo parted /dev/vdg mklabel gpt
sudo parted /dev/vdg mkpart primary 0% 100%

sudo zpool replace tank \
  ee2507fe-0b11-ad4c-b1c5-87e36055410e \
  /dev/disk/by-partuuid/[new-partuuid-of-vdg1]

zpool status tank

Attach Device to Mirror

# Attach new disk to existing single disk (converts to mirror)
sudo zpool attach tank \
  ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K8888888

# Create partition
sudo parted /dev/vdh mklabel gpt
sudo parted /dev/vdh mkpart primary 0% 100%

# Attach using PARTUUIDs
sudo zpool attach tank \
  ee2507fe-0b11-ad4c-b1c5-87e36055410e \
  /dev/disk/by-partuuid/[new-partuuid]

🔍 Pool Maintenance

Check Pool Status

zpool status tank
zpool list tank

Scrub Pool (Verify Data Integrity)

# Start scrub
sudo zpool scrub tank

# Monitor progress
zpool status tank

Export and Import Pool

# Export pool
sudo zpool export tank

# Import pool (whole disk with by-id)
sudo zpool import -d /dev/disk/by-id tank

# Import pool (partitions with by-partuuid)
sudo zpool import -d /dev/disk/by-partuuid tank

# Import without knowing pool name
sudo zpool import

📋 Complete Examples

Example 1: Mirror with Virtual Disks (Most Common for VMs)

# 1. Identify available disks
lsblk

# 2. Verify disks are empty
sudo blkid /dev/vdb /dev/vdc

# 3. Create mirror pool (ZFS creates GPT automatically)
sudo zpool create tank mirror /dev/vdb /dev/vdc

# 4. Check what ZFS created
sudo blkid | grep zfs_member
lsblk

# 5. Create dataset with compression
sudo zfs create -o compression=lz4 tank/data
sudo zfs set mountpoint=/mnt/tank tank/data

# 6. Set permissions
sudo chown manzolo:manzolo /mnt/tank
sudo chmod 775 /mnt/tank

# 7. Test
echo "Hello ZFS" > /mnt/tank/test.txt

# 8. Verify
zpool status tank
zfs list

# 1. Identify disks
ls -l /dev/disk/by-id/ | grep -v part

# 2. Create mirror pool
sudo zpool create tank mirror \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K1234567 \
  /dev/disk/by-id/ata-WDC_WD40EFRX-68N32N0_WD-WCC7K7654321

# 3. Create dataset with compression
sudo zfs create -o compression=lz4 tank/data
sudo zfs set mountpoint=/mnt/tank tank/data

# 4. Set permissions
sudo chown manzolo:manzolo /mnt/tank
sudo chmod 775 /mnt/tank

# 5. Test
echo "Hello ZFS" > /mnt/tank/test.txt

# 6. Verify
zpool status tank
zfs list

# Create RAID-Z1 pool with 3 virtual disks
sudo zpool create tank raidz /dev/vdb /dev/vdc /dev/vdd

# Check status
zpool status tank

# 1. Create partitions
sudo parted /dev/vdb mklabel gpt
sudo parted /dev/vdb mkpart primary 0% 100%
sudo parted /dev/vdc mklabel gpt
sudo parted /dev/vdc mkpart primary 0% 100%

# 2. Get PARTUUIDs
sudo blkid /dev/vdb1 /dev/vdc1

# 3. Create pool (initially with device names)
sudo zpool create -f tank mirror /dev/vdb1 /dev/vdc1

# 4. Transition to PARTUUIDs
sudo zpool export tank
sudo zpool import -d /dev/disk/by-partuuid tank

# 5. Verify PARTUUIDs in status
zpool status tank

# 6. Check UUIDs
sudo blkid | grep zfs_member

# 1. Check status (shows failed partition)
zpool status tank

# 2. Prepare replacement disk
sudo parted /dev/vdg mklabel gpt
sudo parted /dev/vdg mkpart primary 0% 100%

# 3. Get new PARTUUID
sudo blkid /dev/vdg1

# 4. Replace (use PARTUUID shown in zpool status)
sudo zpool replace tank \
  ee2507fe-0b11-ad4c-b1c5-87e36055410e \
  /dev/disk/by-partuuid/[new-partuuid]

# 5. Monitor resilver
zpool status tank

📚 Command Reference

Essential Commands

Device Identifier Commands

💡 Pro Tips

Best Practices

Auto-Import on Boot

sudo zpool set cachefile=/etc/zfs/zpool.cache tank

# Daily snapshots via cron
0 0 * * * /sbin/zfs snapshot tank/data@daily-$(date +\%Y\%m\%d)

# Keep only last 7 days
0 1 * * * /sbin/zfs list -t snapshot -o name | grep daily | head -n -7 | xargs -n 1 /sbin/zfs destroy

# Check for errors
zpool status -x

# Detailed status
zpool status -v tank

# I/O statistics
zpool iostat tank 1

🔧 Troubleshooting

sudo zpool export tank
sudo zpool import -d /dev/disk/by-id tank  # or by-partuuid for partitions

sudo lsof /dev/vdb1
sudo wipefs -a /dev/vdb1  # Clear old filesystem signatures

# Scan for pools
sudo zpool import

# Import by pool ID
sudo zpool import -d /dev/disk/by-id [pool-id]

# Re-import with correct directory
sudo zpool export tank
sudo zpool import -d /dev/disk/by-partuuid tank

🎯 Use Cases

• Home NAS: Mirror pools for media storage with snapshots
• Server Storage: RAID-Z for enterprise data with redundancy
• Backup Systems: Snapshots for point-in-time recovery
• Virtual Machines: Datasets with compression for VM images
• Development: Fast snapshots for testing and rollback

📖 Resources

• OpenZFS Documentation
• Ubuntu ZFS Guide
• ZFS Best Practices

Build robust ZFS pools with persistent device identifiers for reliable storage management!