BTRFS usage- Adding devices, moving data, converting to RAID1, all without rebooting

[oshunluvr]

Thanks for that, jlittle. Always looking for better ways to to things. In this case, it was more the number of files rather size that was at issue. 30,000+ files in 80GB. It did take quite awhile. It's not something I normally do but it was more of a checking of the procedure so that when the inevitable question of veracity came up, I could claim validation. Also, I had deleted my backups to re-configure to RAID1 so I was being cautious.

I'll add your code above to my cheat sheet.

[jlittle]

diff -r @Pictures/ /mnt/pool/@Pictures/ | grep @Pictures/ | awk '{print $4}' > pictures.txt

Using diff -r is handy and convenient, but for terabytes of pictures that will read them all, it might take a while. If you just want to check for existence, diffing a listing would be far faster, say

diff <(cd @Pictures/; find . | sort) <(cd /mnt/pool/@Pictures/; find . | sort)

or for the error prone like me

cd @Pictures
find . | sort  > /tmp/a
cd /mnt/pool/@Pictures
find . | sort > /tmp/b
diff /tmo/a /tmp/b

[GreyGeek]

BTRFS usage- Adding devices, moving data, converting to RAID1, all without rebooting

This is such an excellent post it, and all the other's oshunluver has posted about BTRFS should be grouped into its own forum and pinned.


Sent from my iPhone using Tapatalk

[oshunluvr]

NoWorries: I'm not sure, but I think you might have a couple misconceptions. If it seems I'm talking down to you, please forgive me, but I want to explain a few basic things;

Networked File System (NFS): A file system, or format, resides only on the hard drive it's installed on and does not transmit itself over a network connection. Point being; the file system is irrelevant to the NFS issues you're experiencing. BTRFS will neither add to or reduce your issues in this area. Likely, if you find that removing the NFS shared computer without unmounting causes problems, it is how you have it mounted that is the source of the issue. The most obvious error you might see is "NFS stale file handle." Explore the mounting options soft/hard, bg, and intr to help with this issue. You could also explore a method of auto-mounting and unmounting the share when it is detected or missing.

BTRFS and SNAPSHOTS: BTRFS Snapshots are full copies of the current state of a BTRFS subvolume and may be read/write or read-only. They initially take no space because, rather than actually copying each file, their file "handles" (location and storage information) are stored. When you begin making changes to the source subvolume (the one you took a snapshot of) then the snapshot begins to grow because any file changes that occur to the source are not saved in the snapshot. So for example, you take a snapshot of your home and delete 20 files from it. You would not see any additional available free space because the files were not actually deleted. Their file handles are removed from the source subvolume but not from the snapshot. The snapshot continues to grow as the source subvolume ages (assuming it's in use and changes are being made to it) until it is deleted.

The only place that using a networked device with BTRFS would matter is if you wanted to send (transmit) a BTRFS SUBVOLUME SNAPSHOT over the network. In order to receive the transmitted snapshot, you would either have to have a BTRFS filesystem on the receiving drive or send the snapshot to a file rather than to a subvolume. The disadvantage to sending a snapshot to a file is that you won't be able to enter (view or copy files) from the snapshot until it's restored to a BTRFS file system as a subvolume.

Snapshots and Subvolumes have many fundamental benefits, two of which are: backups and rollbacks. Which is most beneficial depends on your needs.

BACKUPS: Making a backup of your home to a USB disk is a classic use for btrfs snapshot and send/receive. The steps and explanations of them are:

1) Mount the root filesystem that contains the @home subvolume.

The default way Ubuntu and variants like Kubuntu install to btrfs is to create two subvolumes at install time named @ and @home. These subvolumes are then mounted; @ at / and @home at /home. During normal usage you would never actually notice the difference or "see" the subvolumes. In order to make a snapshot of your home subvolume, you would need to mount the primary file system. You could then see and snapshot @ or @home.

2) Make a read-only snapshot of the @home subvolume.

Before a snapshot can be transmitted using send/receive it cannot be in use in any way, otherwise data can be corrupted. The way this is guaranteed is by making the snapshot read-only.

3) Mount the target (USB or whatever) drive.
4) Use btrfs to send/receive the read-only snapshot to the target drive.

This makes a full copy of every file in the snapshot. As you might expect, this may take quite some time depending on the size of the subvolume.

5) Delete the read-only snapshot.

If you don't delete it, it will grow until it reaches the size of the source subvolume.

Backups need not be "sent" anywhere. If you have the drive space simply leaving the snapshot in place will serve as a backup. Sending the backup to another device and then deleting the source snapshot conserves space on the source drive (if needed) and protects you against device failure - just like any other backup method.

One thing to note is the received snapshot is also read-only because it is a copy of the source snapshot. This is usually fine for a backup, but if you wanted to make it read/write for some reason, you simply make a new snapshot of the received subvolume as read-write and delete the read-only one.

ROLLBACKS: A "Rollback" is really just a description of how you use a snapshot to protect yourself from unintended results from an action like an upgrade. The steps and explanations of them are:

1) Mount the root filesystem that contains the @ subvolume.

2) Make a read/write snapshot of the @ subvolume.

It must be read-write so it can be used to replace the source subvolume if needed.

3) Do the action you're protecting yourself from (upgrade, etc.)
If your install is OK after the action:
4) Delete the snapshot.
If your install is NOT OK after the action:
4) Delete the failed subvolume and rename the snapshot to match the deleted subvolume and reboot.

Obviously, if you're working from your install you would have to boot to a LiveUSB install or another install before you could delete and rename the subvolume. Another method to boot to the snapshot be to edit the necessary grub.cfg and fstab to allow booting to the snapshot without renaming, but this is a little complicated so I'm not going into it here. One of the small but mighty benefits to using subvolumes is they can reside on the same filesystem. This means all the UUIDs are identical and booting to a different subvolume can be done without changing UUIDs. The mounting of a subvolume is just a mount option in fstab.

I now have a 256GB SSD attached to my Laptop's USB 3 port, so if I changed to Btrfs and used this to store the rollback files, I understand that I would not loose any free space on my Laptop Home directory. If I then disconnect my Laptop to say go and do a presentation, this arrangement would not cause any problems from disconnecting, making changes to the Laptop stored memory and then reconnecting the SSD to my USB port.

NoWorries, in direct reply to your comments; using the USB drive to store backups is a great way to use BTRFS and if you follow the steps I outlined it wouldn't take any extra space on your laptop drive. However, storing subvolume snapshots on a different drive does not qualify as a "rollback." Once you move the snapshot off the root drive it isn't available to use as a rollback. You would have to go through a full restoration of the snapshot-ed subvolume - just like with any other backup method. The point of making a snapshot for rollback usage is it allows you to quickly and easily restore your system to it source state at the time of the snapshot. Remember a snapshot takes zero space until changes are made to the source subvolume. If you want rollback capability, you'll need to snapshot when you feel it necessary and delete those snapshots when they are no longer needed.

Regarding drive space; if you used BTRFS on your single-drive laptop you wouldn't need to have multiple partitions for Linux. You could change your home, root install, and any other current Linux partitions into subvolumes of a single file system on a single partition. Thus, all the free space is combined and is available to any of the subvolumes. But because the data is segregated by subvolumes, the data remains separate and is accessed just like it is today - only mounted using an option indicating the subvolume name. In effect you would have more usable free space because it isn't artificially divided. This is another amazing benefit to btrfs and it's use of subvolumes.

[NoWorries]

Sorry for the confusion. When I use an Ethernet connection, I always use NFS. And if I mount another computer, I find that I have to make sure that it is unmounted before closing down.

I now have a 256GB SSD attached to my Laptop's USB 3 port, so if I changed to Btrfs and used this to store the rollback files, I understand that I would not loose any free space on my Laptop Home directory. If I then disconnect my Laptop to say go and do a presentation, this arrangement would not cause any problems from disconnecting, making changes to the Laptop stored memory and then reconnecting the SSD to my USB port.

[GreyGeek]

Computer A running Btrfs works normally with itself and Computer B running EXT4 works normally by itself. Having them connected to the same subnet doesn't change anything. Either one could be used as a network server. But, if you are talking about mutiple storage devices attached to a single computer then here is the latest info.

[oshunluvr]

Very impressive with what you have done with Btrfs. I am not such a heavy data user, but I do require to unplug my Laptop and relocate every couple of weeks. Also when I travel, I take my Laptop with me as I can load and edit pictures that I have taken. This saves a lot of time when I get home to finish this process.

Now with this type of activity, does Btrfs make what I do easier as I currently use Ext4?

whether or not btrfs would benefit would depend on your setup and your needs. For example, if your laptop has a separate home partition you could combine all your data -install root and home - into a single btrfs filesystem. This would allow all your free space to be available to both the install and home.

Backups are as simple as snapshot, send, delete snapshot. Restore is just as easy. You can have a simple script do it automatically with cron. Rollbacks are just snapshots. Before you do something potentially dangerous, like an upgrade, simply snapshot first. If all is well, just delete the snapshot. If not, replace your install damaged by the upgrade with the snapshot and delete the non-working install.

Many of the really awesome features are when your using multiple devices like I talk about above but there are many other benefits as well.

[oshunluvr]

I'm unclear what exactly you're doing or what you mean, but as far as I know, btrfs doesn't connect over Ethernet. It's a file system not a network protocol. The connection of a mounted drive is over NFS or SAMBA or?

"Rollbacks" aren't "files" per se, they are snapshots of subvolumes. If you were taking a snapshot for potential rollback use you certainly wouldn't want to transmit it to another computer. It wouldn't be of any use to you there. If you were sending a snapshot for backup then why would not having that computer "attached" during normal use be a threat? It would be no different than using a thumb drive for backup. You'd only need to have it attached to restore from a backup.

[NoWorries]

I cannot remember the last time I needed to do a rollback. Probably because I install a new system every 6 months.

My main concern with Btrfs was related to when you have mounted disks that store all the rollback files and other systems. I have found problems closing down one computer mounted to another which are Ethernet connected. In the case of a system with Btrfs connections to other storage devices, what happens when you remove one computer to use elsewhere and then return? Is the removed computer useable and do you have to unmount and mount to re-establish Btrfs?

[GreyGeek]

BTRFS usage- Adding devices, moving data, converting to RAID1, all without rebooting

My experience with btrfs shows that as far as application use goes the only caveat is that if you use VirtualBox you MUST create your virtual drives with no copy on write, nocow, to avoid corruption of the virtual HD. Other than that Btrfs blows Ext4 away, especially in reversing unwanted changes or restoring to previous conditions. Also, with the default automatic snapshot settings one can eat up HD space rather quickly. On my 625Gb HD using Ext4 with my usual installs I'd have around 525Gb of unused space. With Btrfs using Snapper the "unused" space was 160-200Gb. After neutering Snapper's config settings I used my own script with Snapper and got about 400Gb. Snapshots are virtually empty at first but get bigger as changes are made to the system as old versions of changed files are moved to the snapshot. Rollbacks are a breeze. Oshunluver has several excellent posts documenting the use of Btrfs.


Sent from my iPhone using Tapatalk

[NoWorries]

Very impressive with what you have done with Btrfs. I am not such a heavy data user, but I do require to unplug my Laptop and relocate every couple of weeks. Also when I travel, I take my Laptop with me as I can load and edit pictures that I have taken. This saves a lot of time when I get home to finish this process.

Now with this type of activity, does Btrfs make what I do easier as I currently use Ext4?

[oshunluvr]

Update:

Once all my data folders were sent from the main btrfs pool to the new one, I wanted to be sure everything transferred. So I ran this command on all the subvolumes that I transferred:

diff -r @Pictures/ /mnt/pool/@Pictures/ | grep @Pictures/ | awk '{print $4}' > pictures.txt

This should compare the two subvolumes and list any missing files. None showed up as missing.

Finally, I unmounted the original subvolumes and mounted the new subvolumes in place for a final check of functionality and completeness before removing the original subvolumes.

Looks good so far!

[oshunluvr]

Lol

[GreyGeek]

Indeed!

[dibl]

@oshunluvr, you are the MAN!