Snapraid and MergerFS Setup

Apr 15, 2024

Much of the implementation is taken from these blog posts:

Snapraid

As mentioned in a previous article, my home server uses a Yottamaster 5 Bay Hard Drive enclosure. As of writing, 3 out of the 5 bays are utilized. Each has a 12 TB HDD (not SSD). Two of those are used for data storage, and the last is used as a "parity" drive. This is a concept from multiple different RAID implementations. According to the wikipedia page on parity drives, they are: "...a hard drive used in a RAID array to provide fault tolerance."

A "RAID array" is a combination of hard drives that combines them in some way so that they present to the operating system as a single drive. There are many types of RAID, most of which I am not familiar with.

I am using a sort-of-fake version of RAID called "snapRAID." It is software, rather than hardware, and it's not a "true" RAID in that it is not real-time, like others are. Instead, it is based on snapshots.

You can see more information on the Snapraid site.

MergerFS

MergerFS is a program that allows an operating system to see multiple disks as a single disk. It is not the only such method of combining disks, but it is useful in that you can add or remove disks at will without corrupting files. Unlike some other methods of creating disk unions, individual files are not split across multiple drives.

There are a lot of potential configuration options, but I'm not using most of them.

MergerFS and SnapRAID Implementation

MergerFS Implementation

The implementation is simple. I have three disks. Two of them are combined with MergerFS and mounted at /srv/storage. The third is a snapRAID partity drive.

All of the disks are mounted in fstab. For example, here's a snippet from mine:

# Mergerfs + Snapraid pool
UUID=<uuid goes here> /mnt/disk0 ext4 defaults 0 0
UUID=<uuid goes here> /mnt/disk1 ext4 defaults 0 0
UUID=<uuid goes here> /mnt/parity ext4 defaults 0 0

Later on in fstab, I mount my merged mergerfs filesystem:

# mergerfs pool
/mnt/disk* /srv/storage fuse.mergerfs direct_io,defaults,allow_other,minfreespace=50G,fsname=mergerfs 0 0

This is taking all disks that match the name pattern /mnt/disk* and merging, exposing the combined virtual drive to the OS at /srv/storage.

SnapRAID Implementation

# Defines the file to use as parity storage
# Must not be in a data disk
# Format: "parity FILE_PATH"
parity /mnt/parity/snapraid.parity

# Defines the files to use as content list
# Can store many copies
# Must have at least one copy for each parity file plus one
# More don't hurt
# Can be in the disks used for parity, data, or even boot,
# but each must be on a different disk
# Format: "content FILE_PATH"
content /var/snapraid/.snapraid.content
content /mnt/disk0/.snapraid.content
content /mnt/disk1/.snapraid.content

# Defines the data disks to use
# The order is relevant for parity
# DO NOT CHANGE THE ORDER
# Format: "disk DISK_NAME DISK_MOUNT_POINT"
disk d0 /mnt/disk0
disk d1 /mnt/disk1

# Defines files and directories to exclude
# Remember that all the paths are relative at the mount points
# Format: "exclude FILE"
# Format: "exclude DIR/"
# Format: "exclude /PATH/FILE"
# Format: "exclude /PATH/DIR/"
exclude *.unrecoverable
exclude /tmp/
exclude /lost+found/
exclude *.!sync
exclude .AppleDouble
exclude ._AppleDouble
exclude .DS_Store
exclude ._.DS_Store
exclude .Thumbs.db
exclude .fseventsd
exclude .Spotlight-V100
exclude .TemporaryItems
exclude .Trashes
exclude .AppleDB

That's it!

SnapRAID Sync

SnapRAID does not do anything on its own. You need to set up a sync service, something that runs to synchronize the drives. You can do it manually with snapraid sync, but it's nice to have an automatic method.

I use SnapRAID Runner, which I set up to run daily at 4:30 AM. I used the Root user crontab to do this, since it requires privileges to run.

I also have it sending the results of the sync to my discord server using Apprise. I may write a post on that some day.