Marc's Public Blog - Linux Hacking

How to get Netapp[tm]-like snapshots with BTRFS

Linux did have crappy snapshots if you used LVM, but LVM and LVM2 snapshots were both so bad performance-wise (they were not meant to be long lived, and even a single snapshot would slow your filesysem down significantly, never mind multiple levels).

If you can't use btrfs, but you still want historical snapshots, you should look into LVM thin provisioning which are newer as of kernel 3.4. They are suppposed to be faster for multiple levels of snapshots. Considering how bad LVM2 is, I'm sure they are faster no matter what, but I didn't have a use for them now that I'm using btrfs, so I can't speak of their performance. You can read up more here:

Back to btrfs, I use the recommended layout of putting all filesystems in a subvolume. I this:

After running my script, I get multiple levels of snapshots, I'll show only root here for brevity. With this you can restore files from older versions 3 hours ago, 3 days ago, or 3 weeks ago. Here is the partial output of /mnt/btrfs_pool1:

drwxr-xr-x 1 root root  370 Feb 24 10:38 root

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_daily_20140316_00:05:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_daily_20140318_00:05:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_daily_20140319_00:05:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_daily_20140320_00:05:00

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_hourly_20140316_22:33:00

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_hourly_20140318_00:05:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_hourly_20140319_00:05:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_hourly_20140320_00:05:00

drwxr-xr-x 1 root root  336 Feb 19 21:40 root_weekly_20140223_00:06:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_weekly_20140302_00:06:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_weekly_20140309_00:06:01

drwxr-xr-x 1 root root  370 Feb 24 10:38 root_weekly_20140316_00:06:01

Note that snapshots are not backups, they give you a view into the past if your filesystem hasn't been corrupted and the disk you were using, didn't die.

I then have a cronjob that runs this:

0 * * * * root btrfs-snaps hourly 3 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
2 0 * * * root btrfs-snaps daily  4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
3 0 * * 0 root btrfs-snaps weekly 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'

This is using the script, btrfs-snaps, for which I'll paste a most likely outdated copy here:

#!/bin/bash

# By Marc MERLIN <marc_soft@merlins.org>
# License Apache 2.0.

# This lets you create sets of snapshots at any interval (I use hourly,
# daily, and weekly) and delete the older ones automatically.

# Usage:
# This is called from /etc/cron.d like so:
# 0 * * * * root btrfs-snaps hourly 3 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
# 1 0 * * * root btrfs-snaps daily  4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
# 2 0 * * 0 root btrfs-snaps weekly 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'

: ${BTRFSROOT:=/mnt/btrfs_pool1}
DATE="$(date '+%Y%m%d_%H:%M:%S')"

type=${1:-hourly}
keep=${2:-3}

cd "$BTRFSROOT"

for i in $(btrfs subvolume list -q . | grep "parent_uuid -" | awk '{print $11}')
do
    # Skip duplicate dirs once a year on DST 1h rewind.
    test -d "$BTRFSROOT/${i}_${type}_$DATE" && continue
    echo "Making snapshot of $type"
    btrfs subvolume snapshot "$BTRFSROOT"/$i "$BTRFSROOT/${i}_${type}_$DATE"
    count="$(ls -d ${i}_${type}_* | wc -l)"
    clip=$(( $count - $keep ))
    if [ $clip -gt 0 ]; then
	echo "Will delete the oldest $clip snapshots for $type"
	for sub in $(ls -d ${i}_${type}_* | head -n $clip)
	do
	    #echo "Will delete $sub"
	    btrfs subvolume delete "$sub"
	done
    fi
done