Comparison of Ext4 and Btrfs

[oshunluvr]

So, just for grins, open the /backup folder and navigate to the install backup there, then open the /mnt folder inside the backup of @ and see if anything is in there.

[GreyGeek]

I deleted the snapshot subvolume and all the backups to get a clean /dev/sdb1 (/backup) Btrfs installation. Then I did a balance and a scrub.
After that I created new snapshot ro backups of @ and @home on /mnt and did a send & receive to /backup/snapshots.

Here is the usage data:

:~# btrfs fi usage /mnt
Overall:
Device size: 691.19GiB
Device allocated: 95.07GiB
Device unallocated: 596.12GiB
Device missing: 0.00B
Used: 92.44GiB
Free (estimated): 597.63GiB (min: 299.57GiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 155.95MiB (used: 0.00B)

Data,single: Size:92.01GiB, Used:90.50GiB
/dev/sda1 92.01GiB

Metadata,DUP: Size:1.50GiB, Used:995.28MiB
/dev/sda1 3.00GiB

System,DUP: Size:32.00MiB, Used:16.00KiB
/dev/sda1 64.00MiB

Unallocated:
/dev/sda1 596.12GiB


root@jerry-Aspire-V3-771:~# btrfs fi usage /backup
Overall:
Device size: 698.63GiB
Device allocated: 176.07GiB
Device unallocated: 522.56GiB
Device missing: 0.00B
Used: 174.51GiB
Free (estimated): 523.44GiB (min: 262.16GiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 261.34MiB (used: 0.00B)

Data,single: Size:173.01GiB, Used:172.13GiB
/dev/sdb1 173.01GiB

Metadata,DUP: Size:1.50GiB, Used:1.19GiB
/dev/sdb1 3.00GiB

System,DUP: Size:32.00MiB, Used:48.00KiB
/dev/sdb1 64.00MiB

Unallocated:
/dev/sdb1 522.56GiB

Notice that /backup no longer has a copy of my home account directory. /mnt contains both the OS (@, @home) AND FOUR backup snapshots, two each of @ and @home. /mnt has only the four backup snapshots. Yet, /backup has only 523 Gb of free space and /mnt has 598 Gb. /backup is missing 75Gb of unaccounted for space.

My assumption was that the ro snapshot @home_bkup-2017-04-13, which I send & received to /backup/snapshots contains the full contents of my /home/jerry directory, unlike the ro @home_bkup-2014-04-13 on /mnt.

[mr_raider]

OK, did a test and some research. If you choose manual partitioning and btrfs, the system does indeed create a swapfile. However, btrfs currently cannot properly handle swapfiles so although it's created, it's not being used.

btrfs basically ignores it:

[FONT=monospace][COLOR=#54FF54][B]stuart@zestyVM[/B][/COLOR][COLOR=#000000]:[/COLOR][COLOR=#5454FF][B]~[/B][/COLOR][COLOR=#000000]$ free -m[/COLOR]
            total        used        free      shared  buff/cache   available
Mem:          1997         782         118           4        1096        1050 
Swap:            0           0           0
[/FONT]

There was a bug filed during alpha testing (I think) regarding this.

So currently; If you're proficient enough to want use btrfs, you either do it by manually partitioning and creating a swap partition, or use a different non-btrfs partition to hold the swap file, or don't use swap.

That would explain why my btrfs install in VM kept throwing memory errors and crashing. Good to know.

Sent from my ZTE A2017U using Tapatalk

[GreyGeek]

The only thing that makes sense is there's a nested subvol/snapshot in there somewhere.

You saw the entire file hierarchy in the vdir listing! I like your idea of deleting the snapshots. Tomorrow I'm going to delete all of them on both drives and redo them, after I balance the chunks.

[oshunluvr]

The only thing that makes sense is there's a nested subvol/snapshot in there somewhere.

[oshunluvr]

I did a test here to check my theory on the snapshot thing. I sent snapshot 1 and it consumed about 12GB, then snapshot 2 sent and used about the same. Then I deleted snapshot 2 and re-sent it as an incremental backup. That time it consumed about 300MB.

[oshunluvr]

Yeah, 1 chunk isn't much. I'm not coming up with other ideas at the moment - except maybe deleting the backup subvols and doing the send|receive again to see if you get different results.

[GreyGeek]

$ sudo btrfs balance start -dusage=85 /backup
Done, had to relocate 1 out of 146 chunks

Took about ten seconds.

[FONT=monospace][COLOR=#000000]:[/COLOR][COLOR=#5454FF][B]~[/B][/COLOR][COLOR=#000000]$ sudo btrfs fi usage /backup[/COLOR]
Overall:
    Device size:                 698.63GiB
    Device allocated:            145.02GiB
    Device unallocated:          553.61GiB
    Device missing:                  0.00B
    Used:                        143.90GiB
    Free (estimated):            554.22GiB      (min: 277.41GiB)
    Data ratio:                       1.00
    Metadata ratio:                   2.00
    Global reserve:              197.66MiB      (used: 0.00B)

Data,single: Size:143.01GiB, Used:142.40GiB
   /dev/sdb1     143.01GiB

Metadata,DUP: Size:1.00GiB, Used:765.52MiB
   /dev/sdb1       2.00GiB

System,DUP: Size:8.00MiB, Used:16.00KiB
   /dev/sdb1      16.00MiB

Unallocated:
   /dev/sdb1     553.61GiB

[/FONT]

Freed up about 600Mb.

[oshunluvr]

No, it's chunk balancing. Try this on /backup:

sudo btrfs balance start -dusage=85 /backup

and then compare btrfs usage /backup to the previous one.

[GreyGeek]

Well, that seems odd. A single backup should not be 30% larger than the source. Have you run a balance command to see if the space is freed up?

In order to send|receive a subvolume from one btrfs file system to another the subvolume has to be read-only as this is the only way to transmit the data and insure no corruption during the transmission.

I read that about needing to be "read only" but being the blockhead I am, and thinking that maybe recent updates have abilities not yet documented, I tried it anyway. I haven't done a balance yet. Since my system is only on one device would the "balancing" be between @ and @home?

[oshunluvr]

Well, that seems odd. A single backup should not be 30% larger than the source. Have you run a balance command to see if the space is freed up?

In order to send|receive a subvolume from one btrfs file system to another the subvolume has to be read-only as this is the only way to transmit the data and insure no corruption during the transmission.

[GreyGeek]

Not quite the same situation. Here's my setup:

root@jerry-Aspire-V3-771:~# mount /dev/sdb1 /backup/
root@jerry-Aspire-V3-771:~# vdir /backup/
total 0
drwxr-xr-x 1 root root 21232 Apr 11 22:29 jerry
drwxr-xr-x 1 root root 76 Apr 11 09:47 snapshots

root@jerry-Aspire-V3-771:~# vdir /backup/snapshots/
total 0
drwxr-xr-x 1 root root 256 Apr 11 10:01 @_bkup-2017-04-08
drwxr-xr-x 1 root root 10 Apr 11 10:29 @home_bkup-2017-04-08


root@jerry-Aspire-V3-771:~# mount /dev/sda1 /mnt
root@jerry-Aspire-V3-771:~# vdir /mnt
total 0
drwxr-xr-x 1 root root 256 Apr 8 16:53 @
drwxr-xr-x 1 root root 10 Apr 8 16:54 @home
drwxr-xr-x 1 root root 152 Apr 8 19:37 snapshots


root@jerry-Aspire-V3-771:~# vdir /mnt/snapshots/
total 0
drwxr-xr-x 1 root root 276 Mar 29 11:16 @_bkup-2017-03-29
drwxr-xr-x 1 root root 256 Apr 8 16:53 @_bkup-2017-04-08
drwxr-xr-x 1 root root 30 Mar 29 14:16 @home_bkup-2017-03-29
drwxr-xr-x 1 root root 10 Apr 8 16:54 @home_bkup-2017-04-08
root@jerry-Aspire-V3-771:~#

The @ and @home are only on /mnt. They are not read only, and I got an error when I attempted to send & receive them.
The two bkups in /mnt/snapshots ARE read only and were used to send & receive to sdb1 at /backup/snapshots. The "jerry" on /backups is a Dolphin drag & drop of my account directory.

BTW, the noatime settings would prevent timestamp changes when a file was updated, thus making the modified (or creation) time permanent. Yes?
I can vew "Modified" and "Last Accessed" times using Dolphin, but neither "ls" nor "vdir" show anything but the Modified time. Oh well. I'll change it and see what happens to the speed of my system. If I can't see any improvement I can always remove it.

[oshunluvr]

If I read your posts correctly, I have assumed that your did a send|receive on all 4 subvols and did not do an incremental send of the snapshots. If this is not correct, let me know.

If my assumption is correct, then I believe the extra data usage is is a result of sending the snapshots. A snapshot takes zero space when initially taken and as long as it's on the same filesystem. It only grows as data is changed. But as soon as you did send|receive on the snapshots, the entire data set is sent and it's no longer actually a snapshot - it's now a stand-alone subvolume. This is where an incremental send would have been very useful.

Just so we're both on the same page, an example:

sda contains two subvolumes and a snapshot of each with these imaginary amounts of data:

The subvolumes

12 GB @
20 GB @home

The snapshots of the above subvolumes

2 GB @bckp
5 GB @home_bckp

39 GB in use

Above, @bckp and @home_bckp hold only the changes made to @ and @home. Most the the data is unchanged so resides in @ and @home.

Now we send|receive all four of them from sda to sdb. The new data sizes on sdb might be:

12 GB @
20 GB @home
13 GB @bckp
22 GB @home_bckp

67 GB in use

This is because while residing on sda as snapshots, @bckp and @home_bckp held only the changes that we made to @ and @home. Once you executed send|receive all the data and the changes went to the target file system with each snapshot.

But if we had done an incremental send of @bckp and @home_bckp the amount of used data space would have remained the same. So for example, these would be the commands;

# btrfs send @ | btrfs receive /backup
# btrfs send -p @ @bckp | btrfs receive /backup

and repeat for @home.

Again, assuming you did not do this, you could test my theory by deleting the backup snapshots from /backup (just @bckp and @home_bckp) and then re-sending them using the incremental send.

[GreyGeek]

I was watching Dolphin's free space display values while copying. They match pretty well with "Device unallocated", or "Free (Estimated)".

:~$ sudo btrfs fi usage /
Overall:
Device size: 691.19GiB
Device allocated: 95.02GiB
Device unallocated: 596.16GiB
Device missing: 0.00B
Used: 93.22GiB
Free (estimated): 596.82GiB (min: 298.74GiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 156.98MiB (used: 0.00B)

Data,single: Size:92.01GiB, Used:91.35GiB
/dev/sda1 92.01GiB

Metadata,DUP: Size:1.50GiB, Used:958.12MiB
/dev/sda1 3.00GiB

System,DUP: Size:8.00MiB, Used:16.00KiB
/dev/sda1 16.00MiB

Unallocated:
/dev/sda1 596.16GiB
jerry@jerry-Aspire-V3-771:~$ sudo btrfs fi usage /backup
Overall:
Device size: 698.63GiB
Device allocated: 145.02GiB
Device unallocated: 553.61GiB
Device missing: 0.00B
Used: 143.90GiB
Free (estimated): 554.21GiB (min: 277.41GiB)
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 197.14MiB (used: 0.00B)

Data,single: Size:143.01GiB, Used:142.40GiB
/dev/sdb1 143.01GiB

Metadata,DUP: Size:1.00GiB, Used:765.00MiB
/dev/sdb1 2.00GiB

System,DUP: Size:8.00MiB, Used:16.00KiB
/dev/sdb1 16.00MiB

Unallocated:
/dev/sdb1 553.61GiB

[oshunluvr]

With both drives mounted, do this to both and compare:

sudo btrfs fi usage <MOUNT_POINT>