Marc's Public Blog - Linux Btrfs Blog Posts


All | Aquariums | Arduino | Btrfs | Cars | Cats | Clubbing | Dining | Diving | Electronics | Exercising | Flying | Hiking | Linux | Linuxha | Museums | Public | Rc | Sciencemuseums | Snow | Solar | Trips

I've been using btrfs since 2012, and while as of 2014, it's far from done, it's gone a long way in that time. I thought I'd post some tips and scripts from things I've learned through my own use and sharing with others, hence this page.
More generally, you'll find Btrfs documentation on the btrfs wiki.

>>> Back to post index <<<

2014/03/21 Btrfs Tips: How To Setup Netapp Style Snapshots
π 2014-03-21 00:00 in Btrfs, Linux

How to get Netapp[tm]-like snapshots with BTRFS

Filesystem snapshots are something you'll never want to live without once you've had them. I learned about them in 1997 when I was working for Network Appliance, but unfortunately due to software patents, they managed to pevent most others from enjoying them until more recently.

Linux did have crappy snapshots if you used LVM, but LVM and LVM2 snapshots were both so bad performance-wise (they were not meant to be long lived, and even a single snapshot would slow your filesysem down significantly, never mind multiple levels).

If you can't use btrfs, but you still want historical snapshots, you should look into LVM thin provisioning which are newer as of kernel 3.4. They are suppposed to be faster for multiple levels of snapshots. Considering how bad LVM2 is, I'm sure they are faster no matter what, but I didn't have a use for them now that I'm using btrfs, so I can't speak of their performance. You can read up more here:

  • https://www.kernel.org/doc/Documentation/device-mapper/thin-provisioning.txt
  • https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Logical_Volume_Manager_Administration/thinly-provisioned_snapshot_volumes.html
  • Back to btrfs, I use the recommended layout of putting all filesystems in a subvolume. I this:

  • /mnt/btrfs_pool1 -> actual btrfs filesystem
  • /mnt/btrfs_pool1/root -> gets mounted to / with -o subvol=root
  • /mnt/btrfs_pool1/usr -> gets mounted to /usr with -o subvol=usr
  • /mnt/btrfs_pool1/var -> gets mounted to /var with -o subvol=var
  • After running my script, I get multiple levels of snapshots, I'll show only root here for brevity. With this you can restore files from older versions 3 hours ago, 3 days ago, or 3 weeks ago. Here is the partial output of /mnt/btrfs_pool1:

    drwxr-xr-x 1 root root  370 Feb 24 10:38 root
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140316_00:05:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140318_00:05:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140319_00:05:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_daily_20140320_00:05:00
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140316_22:33:00
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140318_00:05:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140319_00:05:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_hourly_20140320_00:05:00
    drwxr-xr-x 1 root root 336 Feb 19 21:40 root_weekly_20140223_00:06:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_weekly_20140302_00:06:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_weekly_20140309_00:06:01
    drwxr-xr-x 1 root root 370 Feb 24 10:38 root_weekly_20140316_00:06:01

    Note that snapshots are not backups, they give you a view into the past if your filesystem hasn't been corrupted and the disk you were using, didn't die.

    I then have a cronjob that runs this:

    0 * * * * root btrfs-snaps hourly 3 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
    2 0 * * * root btrfs-snaps daily  4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'
    3 0 * * 0 root btrfs-snaps weekly 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'

    This is using the script, btrfs-snaps, for which I'll paste a most likely outdated copy here:

    #!/bin/bash
    

    # By Marc MERLIN <marc_soft@merlins.org> # License Apache 2.0.

    # This lets you create sets of snapshots at any interval (I use hourly, # daily, and weekly) and delete the older ones automatically.

    # Usage: # This is called from /etc/cron.d like so: # 0 * * * * root btrfs-snaps hourly 3 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )' # 1 0 * * * root btrfs-snaps daily 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )' # 2 0 * * 0 root btrfs-snaps weekly 4 | egrep -v '(Create a snapshot of|Will delete the oldest|Delete subvolume|Making snapshot of )'

    : ${BTRFSROOT:=/mnt/btrfs_pool1} DATE="$(date '+%Y%m%d_%H:%M:%S')"

    type=${1:-hourly} keep=${2:-3}

    cd "$BTRFSROOT"

    for i in $(btrfs subvolume list -q . | grep "parent_uuid -" | awk '{print $11}') do # Skip duplicate dirs once a year on DST 1h rewind. test -d "$BTRFSROOT/${i}_${type}_$DATE" && continue echo "Making snapshot of $type" btrfs subvolume snapshot "$BTRFSROOT"/$i "$BTRFSROOT/${i}_${type}_$DATE" count="$(ls -d ${i}_${type}_* | wc -l)" clip=$(( $count - $keep )) if [ $clip -gt 0 ]; then echo "Will delete the oldest $clip snapshots for $type" for sub in $(ls -d ${i}_${type}_* | head -n $clip) do #echo "Will delete $sub" btrfs subvolume delete "$sub" done fi done


    More pages: March 2014 April 2014 May 2014 October 2014

    >>> Back to post index <<<