π
2014-03-21 01:01
in Btrfs, Linux
How to get Netapp[tm]-like snapshots with BTRFS
Filesystem snapshots are something you'll never want to live without once you've had them. I learned about them in 1997 when I was working for Network Appliance, but unfortunately due to software patents, they managed to pevent most others from enjoying them until more recently.
Linux did have crappy snapshots if you used LVM, but LVM and LVM2 snapshots were both so bad performance-wise (they were not meant to be long lived, and even a single snapshot would slow your filesysem down significantly, never mind multiple levels).
If you can't use btrfs, but you still want historical snapshots, you should look into LVM thin provisioning which are newer as of kernel 3.4. They are suppposed to be faster for multiple levels of snapshots. Considering how bad LVM2 is, I'm sure they are faster no matter what, but I didn't have a use for them now that I'm using btrfs, so I can't speak of their performance. You can read up more here:
https://www.kernel.org/doc/Documentation/device-mapper/thin-provisioning.txt
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinly-provisioned_snapshot_volumes.html
Back to btrfs, I use the recommended layout of putting all filesystems in a subvolume. I this:
/mnt/btrfs_pool1 -> actual btrfs filesystem
/mnt/btrfs_pool1/root -> gets mounted to / with -o subvol=root
/mnt/btrfs_pool1/usr -> gets mounted to /usr with -o subvol=usr
/mnt/btrfs_pool1/var -> gets mounted to /var with -o subvol=var
After running my script, I get multiple levels of snapshots, I'll show only root here for brevity. With this
you can restore files from older versions 3 hours ago, 3 days ago, or 3 weeks ago. Here is the partial output of /mnt/btrfs_pool1:
drwxr-xr-x 1 root root 370 Feb 24 10:38 root
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140316_00:05:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140318_00:05:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140319_00:05:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140320_00:05:00
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140316_22:33:00
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140318_00:05:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140319_00:05:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140320_00:05:00
drwxr-xr-x 1 root root 336 Feb 19 21:40 root_weekly_20140223_00:06:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_weekly_20140302_00:06:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_weekly_20140309_00:06:01
drwxr-xr-x 1 root root 370 Feb 24 10:38 root_weekly_20140316_00:06:01
Note that snapshots are not backups, they give you a view into the past if
your filesystem hasn't been corrupted and the disk you were using, didn't die.
I then have a cronjob that runs this:
0 * * * * root btrfs-snaps hourly 3 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
2 0 * * * root btrfs-snaps daily 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
3 0 * * 0 root btrfs-snaps weekly 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
This is using the script, btrfs-snaps, for which I'll paste a most likely outdated copy here:
#!/bin/bash
# By Marc MERLIN <marc_soft@merlins.org>
# License Apache 2.0.
# This lets you create sets of snapshots at any interval (I use hourly,
# daily, and weekly) and delete the older ones automatically.
# Usage:
# This is called from /etc/cron.d like so:
# 0 * * * * root btrfs-snaps hourly 3 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
# 1 0 * * * root btrfs-snaps daily 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
# 2 0 * * 0 root btrfs-snaps weekly 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
: ${BTRFSROOT:=/mnt/btrfs_pool1}
DATE="$(date '+%Y%m%d_%H:%M:%S')"
type=${1:-hourly}
keep=${2:-3}
cd "$BTRFSROOT"
for i in $(btrfs subvolume list -q . | grep "parent_uuid -" | awk '{print $11}')
do
# Skip duplicate dirs once a year on DST 1h rewind.
test -d "$BTRFSROOT/${i}_${type}_$DATE" && continue
echo "Making snapshot of $type"
btrfs subvolume snapshot "$BTRFSROOT"/$i "$BTRFSROOT/${i}_${type}_$DATE"
count="$(ls -d ${i}_${type}_* | wc -l)"
clip=$(( $count - $keep ))
if [ $clip -gt 0 ]; then
echo "Will delete the oldest $clip snapshots for $type"
for sub in $(ls -d ${i}_${type}_* | head -n $clip)
do
#echo "Will delete $sub"
btrfs subvolume delete "$sub"
done
fi
done |