Marc's Public Blog - Linux Hacking

How to manage a btrfs filesystem made out of multiple dmcrypt'ed drives

If you are using raid0, raid1, raid10, raid5, or raid6 with btrfs and you want your filesystem to be encrypted, you need to encrypt each device seperately but later you'll want a script to decrypt all those devices.
This can be done with /etc/crypttab, but I don't personally use it for arrays that I turn off to save power. You can use keyscript= in there to feed a script that will provide the decryption key, but I wrote my own script to tun the disks on, locate them by disk ID, decrypt them, and mount the resulting partition.

If you are planning on using Raid5 or Raid6, you'll also want to read this page.

For the mount to work, you of course have to create the crypted device and filesystem first. Here is a recommended way:

cryptsetup luksFormat -s 256 -c aes-xts-plain64  /dev/sda4
cryptsetup luksFormat -s 256 -c aes-xts-plain64  /dev/sdb4
cryptsetup luksFormat -s 256 -c aes-xts-plain64  /dev/sdc4

cryptsetup luksOpen /dev/sda4 sda4_crypt
cryptsetup luksOpen /dev/sdab sdb4_crypt
cryptsetup luksOpen /dev/sdac sdc4_crypt

mkfs.btrfs -d raid0 -m raid0 -L btrfs_pool /dev/mapper/sd[abc]4_crypt

After reboot, the idea is to avoid the luksOpen steps and adapt to whatever device names those drives could come up under, and this is what the script below does.

Here is the script, start-btrfs-dmcrypt, for which I'll paste a most likely outdated copy here:

#!/bin/bash

# Example script to decrypt a bunch of drives and then mount them as
# part of a btrfs volume.
# 
# By Marc MERLIN <marc_soft@merlins.org> / 2014/04/29
# License: Apache-2.0

# Get these from /dev/disk/by-id
DRIVES="
scsi-SATA_Hitachi_HDS7230_MN5220F323S79K-part1
scsi-SATA_Hitachi_HDS7230_MN5220F325UZMK-part1
scsi-SATA_ST2000DL003-9VT_5YD6MH88-part1
scsi-SATA_ST2000DL003-9VT_5YD70NHX-part1
scsi-SATA_WDC_WD20EARS-00_WD-WMAZA0374092-part1
"

# The label name of your btrfs filesystem (mkfs.btrfs -L btrfs_pool)
LABEL=btrfs_pool

NUMDRIVES=$(echo $DRIVES | wc -w)

die () {
    echo "$1"
    exit 1
}

pwd="$(yourscript that returns crypt key)"
if [ -z "$pwd" ]]; then
    echo -n "Decryption key? "
    stty -echo 2>/dev/null
    read pwd
    stty echo 2>/dev/null
fi
[ -z "$pwd" ]] && die "Didn't get a decryption key"

# Here you can run a command to turn the disks on if they are on an
# external power outlet.
# turn-disks-on-cmd
cd /dev/disk/by-id
for i in 1 2 3 4 5 6 7 8 9
do
    if [ $(ls $DRIVES 2>/dev/null | wc -l) = $NUMDRIVES ]]; then
	break
    fi
    sleep 10
done
# This is useful if the disks were just turned on.
/etc/init.d/smartmontools restart

for i in $DRIVES
do 
    dev=$(ls -l $i | awk '{print $11}' | sed "s#../..#/dev#")
    [ -z "$dev" ]]  && die "Couldn't find device for $i"
    echo "$pwd" | cryptsetup luksOpen "$dev" "crypt_$(basename $dev)" || die "Couldn't decrypt $dev"
    echo "decrypt $dev"
done
btrfs device scan
mkdir -p /mnt/btrfs_pool
mount -v -t btrfs -o compress=zlib,noatime LABEL=$LABEL /mnt/btrfs_pool || die "Couldn't find btrfs $LABEL"