ZFS as your root file system

[GreyGeek]

Unlike Btrfs, it is NOT possible to select ZFS when formatting your "/" partition. So, from the LiveUSB it is not possible to install on a hard drive ZFS running as the root file system, i.e., /boot residing in a ZFS pool.

On the ZFS - Ubuntu Kiki I read yesterday that ZFS CAN be used as the root file system:

Installing ZFS on Ubuntu

The ZFS filesystem is available for Ubuntu as either a FUSE module or a native kernel module. The kernel module is provided by default. To install the user-level tools, simply install:
sudo apt install zfsutils-linux
For all current versions from 16.04 onward.


In addition to be able to have ZFS on root, install:

sudo apt install zfs-initramfs

My assumption is that those two utilities have to be installed on a persistent LiveUSB so they can be used when using the LiveUSB to install ZFS as the root file system, and the rest of the distro, on the target HD. This follows from the instructions given to make ZFS a root file system on Ubuntu 16.04 and 18.04

On Ubuntu, Kubuntu and Neon choosing Btrfs as the root file system is drop dead easy. Just assign Btrfs as the file system for "/" on the partition in the installation section where the HD is prepared. When the installation is complete the "Root_FS" contains @ and @home, which are bound to / and /home in /etc/fstab.

To create an installation where ZFS is the root file system on "/" (which would include /boot) there is a 9 step process which is so complicated that 95% of most Linux users wouldn't begin to attempt it, and of the 5% that do probably less than half would succeed.

Limitations: Only 64b systems, no dual booting, and ZFS must be installed on /dev/sdX, not /dev/sdXn, no trim command, and the cddl license.

The problem: ZFS lacks a systemd mount generator. The problem was discussed extensively two years ago and, apparently, little progress has been made.

Until ZFS gains a systemd mount generator, there are races between mounting filesystems and starting certain daemons. In practice, the issues (e.g. #5754) seem to be with certain filesystems in /var, specifically /var/log and /var/tmp. Setting these to use legacy mounting, and listing them in /etc/fstab makes systemd aware that these are separate mountpoints.

Perusing the installation instructions for either 16.04 or 18.04 I can predict that there will be few people who try to install ZFS as a root file system, even if they have the gazillion GB of RAM necessary to run it at an adequate speed.

[dibl]

Thanks GG, nice report.

Question: What would be the argument FOR a ZFS root partition, over ext4 or btrfs?

[vinnywright]

I will not be trying the ZFS route ,,,,,I am still procrastinating on a full BTRFS system .

at current it's just the 1TB drive and the largest partition (300GB) on the 500GB drive , and I like it .

I see no need to go mucking about learning a "licensed" one when BTRFS is so awesome.

VINNY

[GreyGeek]

Thanks GG, nice report.

Question: What would be the argument FOR a ZFS root partition, over ext4 or btrfs?

I can't think of a valid reason to install ZFS as a root partition, given the elaborate steps necessary to do so now on a Debian or Ubuntu based distro, especially on laptops and desktops as they are commonly used by 95% of people today. But, if ZFS were selectable for a root partition during the install as Btrfs is today, then I'd seriously consider it.

Last year I installed ZFS via zfsutils-linux and played with zpools, snapshots and the like, just to get a feel for it and compare it to Btrfs. Command wise, ZFS is simpler and easier to use and control than Btrfs, but it doesn't offer as much flexibility as Btrfs does, but I'm a Btrfs fanboy. I keep several snapshots of @ and @home on my archive drive. I can rollback to any particular snapshot without losing previous or subsequent snapshots. As I understand it, ZFS snapshots are serial. If I rollback to a specific snapshot those taken after it are lost.

IF I were going to use ZFS I'd first upgrade my RAM to 32GB of ECC stuff, and create the ZFS zpool on a 500GB SSD. That would probably exhaust the SDD rw capacity in no time flat.

However, for my V3-771G Acer (made in 2012) that has an i7 CPU, 6GB RAM, and three internal HDs with NVidia GT 650M GPU as the video chip, using Btrfs is a perfect fit.

There are some distros that install ZFS on a root file system. One OS folks talk about a lot in regard to ZFS is the BSD TrueOS.

[GreyGeek]

Here is a recent video of a fellow playing around with three 16GB USB sticks as he creates as zpool and plays with it.
It shows how really robust ZFS is.

[simonsaysthis]

Thanks GG, nice report.

Question: What would be the argument FOR a ZFS root partition, over ext4 or btrfs?

From hearsay I believe ZFS has some distinct advantages in server space. Especially in a production SAN/NAS environment performance is meant to be stellar, I think Canonical really really wanted it but there were some licensing issues.

[dibl]

I recall a friend mentioning that FreeNAS installs a ZFS filesystem by default when you use it to set up a NAS.

Personally, I view the OS as an easily-replaced component of my computer system. Indeed, I had to replace it and the SSD it was on when the SSD crapped out in 2013. But my 600+ GB of user data are mostly non-replaceable if lost, so that is where my storage security resources go. A pair of WD1000 hdds, BTRFS filesystem on top of them in default configuration (mirrored metadata, striped data), periodic full backups to spare hdds, occasional backups to a cloud storage service, etc.

[GreyGeek]

I write a lot about the ease of snapshotting and rolling back with Btrfs (I've done it twice in the last 24 hours) BUT, and it is a BIG BUT, snapshots are not backups unless you use Btrfs's send & receive commands to move a snapshot pair to an external HD that can be moved offsite.

Btrfs, running as a root file system, is drop dead easy to use. All commands begin with "sudo btrfs ...". After installation the most basic and often repeated tasks are to:
1) take a snapshot of @ & @home before any major change (usually an update, or installing a large or complex app, like I did with P2P software last year),
2) Sending & receiving those snapshots to external or remote drives,
3) Rolling back to previous snapshot pairs.

Things rarely done include resizing the pool by expanding or shrinking, scrubbing, balancing, or checking -- like fsck, the only command that has to be done from a LiveUSB while the system is shut down. I keep on hand a persistent LiveUSB of Neon created by installing Btrfs as the root file system. Earlier this spring, after three years and changing from a one HD single to a two HD RAID1 to a two HD RAID0 to a one HD single again with two archival HDs, I did a check. It produced no errors. I've done balancing and scrubbing a couple times.
That's about the sum total of what I do with Btrfs.

One serious advantage of Btrfs, besides being a root file system on *ubuntu's, is the ability to make snapshots of individual subvolumes and restore to any particular snapshot without destroying any snapshots made later. Say I have 10GB of installed tools and files for a project. I complete the project and don't want them to take up space on my pool any more. If I installed those tools in their own subvolume, say /home/jerry/data, and make a dated snapshot of it, @data20180525, I can then delete the @data subvolume and comment out the line in fstab that automatically mounts @data to /home/jerry/data. I go on working away and several days later I realize that I need to roll @ and @home back to 20180428. I take a dated snapshot of @ and @home and then roll them back to 20180428. No problem. When I'm done I roll @ and @home forward to the snapshots I just took before the rollback. The @data20180525 is still there, untouched. Later, if I needed to reaccess that project I just reverse the snapshot command, take the # from the line in fstab, and mount it. Nothing else is changed or lost.

ZFS has two tools, zfs, which controls directories and files, and zpool, which interacts with devices to make and controls pools.

ZFS obviously takes snapshots but:

• A data set cannot be destroyed if snapshots of the data set exist.
• If clones have been created from a snapshot, they must be destroyed before the snapshot can be destroyed.

Not really biggies.
All in all, ZFS has much more flexibility than Btrfs, is more difficult to learn, IMO, but for servers it is the FS of choice, also IMO. IF a systemd mount generator ever gets made for ZFS it would become a serious challenge to Btrfs on *ubuntu's. I, for one, would probably switch, because ZFS is about 3X faster than Btrfs.

Ubuntu creates just two subvolumes during installation: @ and @home. OpenSUSE creates about a dozen! The best practice with ZFS appears to me to follow in SUSE's footsteps and create a zpool and then create datasets for every directory: /, /boot, /opt, /src, /home/user, /mnt, /media, etc... The problem with that is that when you install applications files are often distributed across several directories (datasets), just creating snapshots for a few of them and at various times would make it hard to restore a particular system wide point in time. As I wrote before, ZFS's commands are simple, but there are a LOT more of them and they have a LOT of parameters and switches.

Meanwhile, I am going to resize the btrfs pool on my sdb drive so I can create 300-400GB of free space and install ZFS on it to play with.