Beginner's guide to BTRFS

[Thomas00]

With your mouse highlight the 47a4794f-b168-4c61-9d0c-cc22f4880116 part of the sda1 UUID and use right mouse copy to copy it. Then, in the Konsole, enter
mount /dev/disk/by-uuid/uuidhere /mnt
replacing "uuidhere" with 47a4794f-b168-4c61-9d0c-cc22f4880116. That mounts the drive at /mnt and /mnt becomes the <ROOT_FS> of your Kubuntu BTRFS installation. It is the top of the tree. @, @home and all other subvolumes you may add are under <ROOT_FS>. Notice that your system and desktop are still running. You can switch to them and do stuff if you need to. You are doing live maintanence on your system!

If you do
vdir /mnt
you will see
/mnt/@
/mnt/@home

Hi GreyGeek, your advice in this thread worked perfectly when I converted my laptop to btrfs, but right now I am converting my homeserver which after some back and forth ended up to be a fresh install of Ubuntu 18.04.
I followed your advice but when I mount / like proposed above I don't get @ and @home in /mnt but the / filesystem. or in other words, ll / and ll /mnt lead to the same result.

blkid gives me:
/dev/sde2: UUID="9c3b40b6-c716-11e8-a25e-bcaec52979b7" UUID_SUB="9de1419d-bce2-4df0-881a-18b321c698ed" TYPE="btrfs" PARTUUID="73f89684-7bb9-4719-aca5-235a51b8dd2c"

Then I do a:
mount /dev/disk/by-uuid/9c3b40b6-c716-11e8-a25e-bcaec52979b7 /mnt

Followed by:
root@homeserver:~# vdir /mnt
total 16
drwxr-xr-x 1 root root 2430 Oct 3 14:38 bin
drwxr-xr-x 1 root root 620 Oct 3 14:39 boot
drwxr-xr-x 1 root root 114 Jul 25 23:03 dev
drwxr-xr-x 1 root root 3158 Oct 3 17:03 etc
drwxr-xr-x 1 root root 12 Oct 3 14:18 home
lrwxrwxrwx 1 root root 33 Oct 3 14:16 initrd.img -> boot/initrd.img-4.15.0-36-generic
lrwxrwxrwx 1 root root 33 Jul 25 23:01 initrd.img.old -> boot/initrd.img-4.15.0-29-generic
drwxr-xr-x 1 root root 462 Jul 25 23:01 lib
drwxr-xr-x 1 root root 40 Jul 25 22:58 lib64
...

No @ and @home
Any idea what I am doing wrong?

[GreyGeek]

The following exchange was noticed on a YouTube video about BTRFS. Frank Barcenas asked "Why would anyone choose BTRFS over ZFS?"

Steve Bergman
Frank Barcenas - Because filesystems should not have memory requirements measured in multi-gigabytes? Because ZFS is slow? Because ZFS is over-hyped? Because ZFS is incompatible with GPLv2? Because it's a piece of alien, out of tree software that you have to update manually if you want to keep up with security updates? Because it's a rampant layering violation? Because I don't really give an FF about any of its vaunted, but mostly not so useful "features"? Because if will still be a long time before 64 bit filesystems like XFS are not far more than just enough? There is a quick, partial list off the top of my head. I didn't go into why ext4 is more reliable in the real world. And oh yeah. XFS and ext4 are supported.

I might add that BTRFS is selectable as a root file system during an Ubuntu/Kubuntu/Neon install, but ZFS is not. One has to jump through NINE major steps, each with several sub-steps, to create a ZFS root file system, including having to use chroot to set things up, and vfat on a UEFI partition for booting. Only a fraction of 1% of Linux users would attempt the following steps:

https://github.com/zfsonlinux/zfs/wi...04-Root-on-ZFS

[GreyGeek]

Can you please clarify this to me:

1) Does btrfs scrub only makes sense with RAID or backup on external drive can be enough to correct damaged files? If scrub discovers corrupt file will it be enough to recover from backup? After all it doesn't work on disk level, it works on file level. If scrub command shows errors one of the sources of them may be corrupt disk and other measures need to be taken, like checking disk for errors, correct?

Like oshunluver and the Btrfs man pages say,

"btrfs scrub is used to scrub a btrfs filesystem, which will read all data and metadata blocks from all devices and verify checksums. Automatically repair corrupted blocks if there’s a correct copy available.

Note: Scrub is not a filesystem checker (fsck) and does not verify nor repair structural damage in the filesystem."

2) Do I assume correctly that incremental backup on btrfs adds only new files from source subvolume to backup?
Or does it also remove from backup when they where removed from source?

Does incremental backup makes sense for root subvolumes where data are not only added but removed or changed every system update, every new program install or uninstall? Would't make more sense to delete existing backup and make a new one once upon a time, let's say when major changes are done to system or when full backup was done a long time ago and almost all files on system has new versions like in comparison of first LTS release 16.04 and 16.04.4?

I never make incremental backups of @ or @home, and when I do make snapshots of them they are always matched pairs, i.e., they have the same date: @YYYYMMDD and @homeYYYYMMDD.

At the level of the <ROOT_FS>, where @ and @home reside, I also have @Data, which is bound in fstab to /home/jerry/Data, where I keep my important data. I have been considering replacing /home/jerry/Downloads and /home/jerry/Documents and replacing them with them with @Downloads and @Documents, since that is what is mainly in @Data, but time will tell.

So, I make a -r snapshot of @Data labeled @Data20180902. I send it to my backup storage. The next day, after adding C & D and deleting A & B, I create a -r snapshot of @Data labeled @Data20180903. @Data20180902 contains A & B but NOT C & D. @Data20180903 contains C & D but NOT A & B. Notice that I have not sent @Data20180903 to the backup storage.

Now I do an incremental backup.
btrfs send -p @Data20180902 @Data20180903 | btrfs receive /backup

The receive part of the command takes the @Data20180902 snapshot, which is already on /backup, and creates a copy labeled @Data20180903 on /backup. Everything in @Data20180902 does NOT have to be resent since the receive command has a copy which it used to create @Data20180903. The send command sends only the difference data: A & B have been deleted and C & D have been added. Thus, the receive command makes its copy of @Data20180903 look like the snapshot @Data20180903 on the send side, minus A & B but containing C & D, and very little actual data has to be transmitted, making it very fast. Typically, my @Data subvolume takes about 15 minutes for a full send & receive. Using incremental sending reduces that time to a few seconds to a couple minutes, depending on the changes.

Now, the next day, 20180904, a snapshot is made and the -p command is used to send the difference between @Data20180903 and @Data20180904, and so on, day after day.

3) Does it make sense to use autodefrag option in fstab for @ and @home subvolume, as well as for @Data where I keep all else (pics, documents, etc. but not virtualization images or big databases). It is a normal home PC mostly used for browsing Internet. Is it better to defragment manually with command once upon a time?

I use: defaults,noatime,space_cache,autodefrag options for each subvolume.
I have read this article but I don't use nodatacow nor compression.

Here are my SSD settings for @ in fstab:
btrfs defaults,ssd,autodefrag,noatime,subvol=@ 0 1

I've read somewhere, but can't find it right now, that only the first row of settings is used for all rows. Different settings for @home or other partitions are ignored.

[oshunluvr]

1) Does btrfs scrub only makes sense with RAID or backup on external drive can be enough to correct damaged files? If scrub discovers corrupt file will it be enough to recover from backup? After all it doesn't work on disk level, it works on file level. If scrub command shows errors one of the sources of them may be corrupt disk and other measures need to be taken, like checking disk for errors, correct?

"Scrub" is not a "corrupt file" checker. It is a tool to check the filesystem (bad sectors, etc.) not the files. The tool to check files is "btrfs check". The recommendation is to run scrub periodically, monthly or so. btrfs check is considered the "last resort" of data recovery. The preferred method for disaster recovery is mounting read-only and using "btrfs restore". In short, if you detect a bad file, the data is likely already damaged which is why we do backups.

RAID or external devices have no impact on whether or not you should run scrub or or any other maintenance. They're all the same to the tools.

For the record: I have been using btrfs since tools version 0.19 (that's not a typo - it was pre-version 1) I have had four files corrupted that shared a sector that was damaged by a failing drive controller. This was in 2014 so several of the tools we have now were in their infancy or unreleased. I was unable to recover or delete them (damage was to the inode) but I was able to rename the damaged files and restore the originals from a backup into the same location and continue using the system - after I replaced the controller of course.

2) Do I assume correctly that incremental backup on btrfs adds only new files from source subvolume to backup?
Or does it also remove from backup when they where removed from source?

You're kind of half-way there in your understanding of this.

Backups - incremental or not - are files-to-file copies of the current state of the filesystem. An incremental backup will not include any files that were deleted because, well, they were deleted. However, as long as you retain the previous backup(s) the deleted files will exist there unless and until you delete the previous subvolume.

Example working with subvolume "@home";

• You backup monthly on the first of every month.
• First backup sent is named "@home_june"
• Then you delete and add some files, etc.
• Then you do an incremental backup and send it as "@home_july"
• Then you delete and add some files, etc.
• Then you do an incremental backup and send it as "@home_august"

On your backup file system you now have three backups, "@home_june, "@home_july, and "@home_august". Any files deleted after June 1st are not in "@home_july" or "@home_august". Likewise, files deleted after July 1st are still in "@home_june" and "@home_july" but not "@home_august". At this point none of the files you deleted in June or July are actually gone, they're in the previous snapshots. If you determine that keeping "@home_june" is no longer necessary and you delete it, the files deleted during and prior to June are now actually, finally deleted.

Remember, each of these subvolumes share data space. So any files that "exist" in two or all three are actually on the hard drive only once. This is true of all snapshots.

The send|receive commands are a way to transfer the location of a snapshot. The incremental feature allows you to send only the difference between the snapshot and the subvolume thus save time and possible data space. You need not keep the previous subvolumes for a "standard" backup but if you want to retain some level of "rollback" among your backups, incremental backups are the way to do it.

Does incremental backup makes sense for root subvolumes where data are not only added but removed or changed every system update, every new program install or uninstall? Would't make more sense to delete existing backup and make a new one once upon a time, let's say when major changes are done to system or when full backup was done a long time ago and almost all files on system has new versions like in comparison of first LTS release 16.04 and 16.04.4?

Whether or not it makes sense is an individual decision. I can make a case for either. For example, I want to upgrade my video driver. I take a snapshot, run the upgrade and test. If the driver installs and works OK, I might delete the snapshot right then and there. For periodic snapshots, I would keep some number that makes sense to me, like taking a weekly snapshot and keeping the last 5 of them. But I would agree that a backup of the root subvolume that's very old would be of little value.

The real answer lies in your backup routine and your tolerance for potential loss. Losing the root subvolume is more a function of having to redo upgrades, installations of new software, and any system setting changes you made, vs. losing home data may mean irrevocable loss of many work hours or important or sensitive information. I think the solution lies in blending the two actions (snapshots and backups) to achieve a comfortable level of data security with a reasonable workload.

You might want to do daily snapshots of both root and home, but also situational snapshots of root. You also might think a weekly backup of home is necessary but a monthly backup of root is OK. Obviously, storage consideration is a factor as well. There's a lot of personal variables that only you can answer.

[gnomek]

Can you please clarify this to me:

1) Does btrfs scrub only makes sense with RAID or backup on external drive can be enough to correct damaged files? If scrub discovers corrupt file will it be enough to recover from backup? After all it doesn't work on disk level, it works on file level. If scrub command shows errors one of the sources of them may be corrupt disk and other measures need to be taken, like checking disk for errors, correct?

2)

It then begins to populate @YYYYMMD2 with the data not already in @YYYYMMD1, the incremental data.

Do I assume correctly that incremental backup on btrfs adds only new files from source subvolume to backup?
Or does it also remove from backup when they where removed from source?

Does incremental backup makes sense for root subvolumes where data are not only added but removed or changed every system update, every new program install or uninstall? Would't make more sense to delete existing backup and make a new one once upon a time, let's say when major changes are done to system or when full backup was done a long time ago and almost all files on system has new versions like in comparison of first LTS release 16.04 and 16.04.4?

3) Does it make sense to use autodefrag option in fstab for @ and @home subvolume, as well as for @Data where I keep all else (pics, documents, etc. but not virtualization images or big databases). It is a normal home PC mostly used for browsing Internet. Is it better to defragment manually with command once upon a time?

I use: defaults,noatime,space_cache,autodefrag options for each subvolume.
I have read this article but I don't use nodatacow nor compression.

[oshunluvr]

I've never actually tested this as in trying to boot to it. You should try it out and let us know if it works!

Well, I just tried the sed commands and rebooted to the new drive and it worked like a charm!

One thing I did that I didn't state before was manually go into grub.cfg on the new drive and remove "--hint-bios=hd2,gpt3 --hint-efi=hd2,gpt3 --hint-baremetal=ahci2,gpt3" because I was unsure if the BIOS would change the device order when I booted to the second drive. I removed this twice - once the the first part of grub.cfg and once in the first menuentry that I was going to boot to. Once booted, rerun update-grub and all should be well.

Since I intend to remove the other drive, I also used sed to change the UUIDs of the swap partition and the secondary EXT4 partition I keep my virtual machines on.

[GreyGeek]

Here is another good source for explaining how to chroot into @ on a device with a corrupted grub.

http://logan.tw/posts/2015/05/17/gru...t-file-system/

However, for all problems except grub corruption using chroot isn't necessary.
Also, IMO, creating snapshots under / or /home (like Snapper does) is not a good idea. If @ or @home becomes corrupted you may not be able to access your snapshots in order to recover. Storing your snapshots under <ROOT_FS> (i.e., /mnt or /subvol, for example) allows you to mount your /dev/disk/by-uuid/uuuiiiiddd to /mnt, giving you access to @, @home and the contents of /mnt/snapshots. IF /dev/disk/by-uuid/uuuiiiddd doesn't mount your problems may be hardware, not software.

[gnomek]

Thank you. I followed @oshunluvr method and it works. But I had to reinstall grub. I don't know why. When I started machine I didn't even met grub rescue, only pulsating _.

I followed method from this site:
but instead of grub2 I used 'grub-install'.

I discovered that it is possible to chroot to @ subvolume.

[GreyGeek]

Sorry that I didn't explain my intentions. I was thinking about the case when I move my system from one disk to another (larger) or when disk is dead and I want to use backup to move @_subvolume and @home_subvolume to new disk. To be able to start system from this new disk I need to change UUID in fstab. With chroot it was easy. Now I don't know how to do it.

On backup I can't edit files because they are read only.

This may be useful information and worth adding to guide. After that you can delete my posts or move as a separate topic.

The mv command can only be used to move snapshots, which are also subvolumes, within the <ROOT_FS> (in my case /mnt) and NOT outside of or between <ROOT_FS>'s. Moving snapshots to another <ROOT_FS> requires the use of the btrfs send & receive command.

Snapshots can be sent to EXT4 and other non Btrfs filesystems by using the -f flag, but be forewarned that I have never tried this procedure. All I've presented below is my interpretation of the btrfs send and receive man pages.
btrfs send -f some_asciifile.txt snapshotname | btrfs receive /destinationdirectory

A copy of the snapshotname will appear on /destinationdirectory as some_asciifile.txt, which will contain what was the ascii stream of the snapshot.
Reversing the send from the destination drive:
btrfs send some_asciifile.txt | btrfs receive /mnt/snapshotname



Read only snapshots can be converted to read-write using
btrfs property set -ts nameofsnapshot ro false
and back to read only
btrfs property set -ts nameofsnapshot ro true

[oshunluvr]

Sorry that I didn't explain my intentions. I was thinking about the case when I move my system from one disk to another (larger) or when disk is dead and I want to use backup to move @_subvolume and @home_subvolume to new disk. To be able to start system from this new disk I need to change UUID in fstab. With chroot it was easy. Now I don't know how to do it.

On backup I can't edit files because they are read only.

This may be useful information and worth adding to guide. After that you can delete my posts or move as a separate topic.

OK, so here's the commands I use in my script. I didn't post the script because its very long and is intended to do a full automated backup and it very specific to my system.

source_UUID=`lsblk -no UUID "$source_device"`
backup_UUID=`lsblk -no UUID "$backup_device"`

sed -i "s/$source_UUID/$backup_UUID/g" "$backup_mount""$distro"/etc/fstab
sed -i "s/$source_UUID/$backup_UUID/g" "$backup_mount""$distro"/boot/grub/grub.cfg

Replace "$source_device" and "$backup_device" with your mounted btrfs file system device locations (/dev/sda3, etc.)
Replace "$backup_mount""$distro" with the subvolume path and name.

To expand, assume these are your btrfs file systems. Remember you have to mount your root BTRFS file system to get to your subvolumes, so for this example I used "/subvols";

             UUID                             MOUNT POINT  
Source  8f0c1661-4e84-4512-b875-23bcfd5be1d8  /subvols    
Backup  96e0e474-bd09-4389-b481-6b3dc409f913  /mnt/backup

Assuming you use the default "@" as your install root subvolume, your script might look like;

btrfs subvolume snapshot -r /subvols/@ /subvols/@ro
btrfs send /subvols/@ro | btrfs receive /mnt/backup
btrfs subvolume snapshot /mnt/backup/@ro /mnt/backup/@
btrfs subvolume snapshot delete -c /mnt/backup/@ro
btrfs subvolume delete -c /subvols/@ro
sync
sed -i 's/8f0c1661-4e84-4512-b875-23bcfd5be1d8/96e0e474-bd09-4389-b481-6b3dc409f913/g' /mnt/backup/@/etc/fstab
sed -i 's/8f0c1661-4e84-4512-b875-23bcfd5be1d8/96e0e474-bd09-4389-b481-6b3dc409f913/g' /mnt/backup/@/boot/grub/grub.cfg

Breakdown by line:

1. Takes a read-only snapshot
2. Sends it to the backup file system
3. Re-snapshots the read-only backup to read-write and uses the subvolume original name
4. Deletes the read-only snapshot on the backup file system
5. Deletes the read-only snapshot on the root file system
6. Sync the drives to flush the cache
7. Replaces the UUIDs in fstab in the backup subvolume
8. Replaced the UUIDs in grub.cfg in the backup subvolume

You'd need to run it as root obviously.

"sed" is a stream editor. The "-i" switch does "replace in-place" of the two fields as 's/<replace this>/<with this>/g'. The trailing "g" means "act globally instead of just on the first occurrence".

I've never actually tested this as in trying to boot to it. You should try it out and let us know if it works!

[GreyGeek]

....

1. If you are doing regular backups, learn to do incremental send|receive as it can save a ton of time and drive space.

Yesterday I added a section on incremental backups and explained how they work.

[oshunluvr]

Sorry that I didn't explain my intentions. I was thinking about the case when I move my system from one disk to another (larger) or when disk is dead and I want to use backup to move @_subvolume and @home_subvolume to new disk. To be able to start system from this new disk I need to change UUID in fstab. With chroot it was easy. Now I don't know how to do it.

On backup I can't edit files because they are read only.

This may be useful information and worth adding to guide. After that you can delete my posts or move as a separate topic.

I do this (boot a subvolume from a different disk) using a sed one-liner to change the UUID in grub.cfg and fstab. I'll dig it up when I have a sec and post it.

[oshunluvr]

I run things a little different than GG.

1. "Snapshots" are exactly that so don't need to be relocated at all unless you're intent on making an actual backup. Snapshots are very useful for rollback protection, etc., when locally stored and read-write rather than read-only.
2. If you are doing regular backups, learn to do incremental send|receive as it can save a ton of time and drive space.
3. TRIM is best done with Anacron and no more than monthly if you use the file system daily and/or run at 70-80% capacity regularly. Otherwise semi-annually is fine.
4. I recommend the "autodefrag" mount option because, well, it's there and it can't hurt.

Shameless plug: My Dolphin BTRFS Subvolume Service Menu

[oshunluvr]

I know absolutely nothing about UEFI installs. Never done one. There are several on this forum who are knowledgeable in both Btrfs and UEFI.

Perhaps Oshunluver will edit the work and add information related to EUFI installs. Oshunluver?

No UEFI here

[gnomek]

I know of NO reason to run chroot on the <ROOT_FS>, and certainly not to change the UUID. The UUID is generated automatically and the chances of two HD's having the same UUID are essentially zero.

Sorry that I didn't explain my intentions. I was thinking about the case when I move my system from one disk to another (larger) or when disk is dead and I want to use backup to move @_subvolume and @home_subvolume to new disk. To be able to start system from this new disk I need to change UUID in fstab. With chroot it was easy. Now I don't know how to do it.

On backup I can't edit files because they are read only.

This may be useful information and worth adding to guide. After that you can delete my posts or move as a separate topic.