BTRFS - caution before using ZSTD compression - boot problems

[oshunluvr]

A new and improved form of compression when using btrfs has made it into the btrfs-tools called "zstd".

Note: GRUB cannot read this form of compression as of yet so if your boot folder is in a btrfs file system compressed with zstd you will not be able to boot.

I discovered this as I was moving my subvolumes - which includes my @grub subvolume - to my new drive. I had mounted it with ZSTD compression instead of my usual LZO. This rendered my new device unbootable immediately. Digging into the grub console and I discovered an error about "0x3" no readable and that turned out to mean the compression type I had used.

I was able to recover by booting to my older SSD that was still using LZO and mounting the new file system with LZO, then copying the files in @grub to a new subvolume and doing the same with the boot folder of @KDEneon. Right now I'm posting from my new drive and it's new btrfs filesystem.

Until this is addressed (the developers are aware), only use ZSTD on non-booting file systems.

That is all. As you were...

[vinnywright]

interesting ,,,higher compression than lzo and almost as fast to compress and decompress .

will start watching this one , thanks for the heads up @oshunluvr

VINNY

[oshunluvr]

In the tests I've read the benefits are slight (as they typically are with this kind of thing). I tried it as an experiment when transitioning to my new blazin' fast M.2 drive but ran myself into a brick wall! Easy fix at least.

[YAFU linux]

Hi.
I am interested in trying KDE Neon and with btrfs filesystem. I'm interested mostly in testing compression feature, I'm not interested in snapshots.
Looking for which compression method was convenient, I found this:
https://www.phoronix.com/scan.php?pa...compress&num=1

But I also find this thread about problems with GRUB and ZSTD, So for now I'm not going to use it.
I ask some questions here.
Is it possible to install KDE Neon in only one Btrfs partition? (this is, /home and all in only one partition. It's what I've always done with any file system)
Should the compression method be selected during or after the installation of KDE Neon?
Thanks.

[oshunluvr]

Hi.
I am interested in trying KDE Neon and with btrfs filesystem. I'm interested mostly in testing compression feature, I'm not interested in snapshots.
Looking for which compression method was convenient, I found this:
https://www.phoronix.com/scan.php?pa...compress&num=1

But I also find this thread about problems with GRUB and ZSTD, So for now I'm not going to use it.
I ask some questions here.
Is it possible to install KDE Neon in only one Btrfs partition? (this is, /home and all in only one partition. It's what I've always done with any file system)
Should the compression method be selected during or after the installation of KDE Neon?
Thanks.

The installer, Ubiquity,does not have user-level access to the subvolume structure during installation. It will, by default, create two subvolumes on the btrfs file system @ and @home. You must re-configure after installation.

However, this is rather trival, simply move your home folder to the @ subvolume and unmount, remove the fstab reference, and delete @home. Won't take 2 minutes if you know what you're doing. I'm not really sure what the benefit of this is to you though. Subvolumes and snapshots are the primary reasons to use btrfs, so why bother?

AFAIK, re. compression: It's done on a file-by-file basis and not at filesystem creation time but at mount time. So if you create a btrfs file system and mount it with LZO compression, every file added after that will be LZO compressed. If you were to remount with ZLIB or ZSTD compression, the files that had been compressed with LZO will remain in that state until they are written again. I think if you remounted and balanced, you could force a re-compression of every file but I'm not sure. Compression is the last reason I use btrfs so I haven't but much into researching it.

The ZSTD issue is a grub problem. Until it is addressed, the files in your /boot folder must not be compressed using ZSTD.

If you're interested in testing btrfs compression, I suggest not worrying about the installation at all. Create four new subvolumes after installation, mount each of them with each of the 3 compression types and 1 with no compression. Then dump a bunch of data into each of them.