No announcement yet.

btrfs replace consideration

  • Filter
  • Time
  • Show
Clear All
new posts

    btrfs replace consideration

    I've been trying to wrap my head around the btrfs replace command. My relevant drives are as follows:

    root@GreyGeek:/# fdisk -l

    The source drive:
    Disk /dev/sda: 465.76 GiB, 500107862016 bytes, 976773168 sectors
    Disk model: Samsung SSD 860
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disklabel type: gpt
    Disk identifier: CF604933-F7D3-49B1-B0E3-BC7C564CEED9

    Device Start End Sectors SizeType
    /dev/sda1 2048 585727 583680 285M EFI System
    /dev/sda2 585728 4491263 3905536 1.9G Linux swap
    /dev/sda3 4491264 976771071 972279808 463.6G Linux filesystem

    The target drive:
    Disk /dev/sdb: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
    Disk model: SSD 870 EVO 1TB
    Units: sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 33553920 bytes
    Disklabel type: gpt
    Disk identifier: 11A30D08-8E0C-204C-BFEA-B80B90C991FE

    Device Start End Sectors SizeType
    /dev/sdb1 2048 1953523711 1953521664 931.5G Linux filesystem

    and my current btrfs partition on /dev/sda3 shows:
    root@GreyGeek:/# btrfs fi show /
    Label: none uuid: 8031d05e-03c9-4012-ab07-89ef71983232
    Total devices 1 FS bytes used 132.04GiB
    devid 1 size 463.62GiB used 142.07GiB path /dev/sda3

    Notice that my source drive (/dev/sda) has three partitions and the btrfs partition is on sda3. As root I always mount it to take snapshots using:
    mount /dev/disk/by-uuid/8031d05e-03c9-4012-ab07-89ef71983232 /mnt

    Also, as you can see in the fdisk listing, I've prepared the new 1TB drive by creating a btrfs partition: /dev/sdb1. There are no other partitions on that SSD.

    The command to replace /dev/sdb1 with the contents of /dev/sda3 would be:
    btrfs replace start 1 /dev/sda3 /mnt/

    I'm using the only guide I found on the web explaining how to use the btrfs replace command, which I found here. All other explanations I've found on the web merely reword the man pages but give no useful examples or poignant advice.

    What I do not understand, nor find on the Internet, is how the sda1 and sda2 partitions get copied to sdb, because without them booting sdb would be impossible, and I'd be forced to simply replace my current SSD with the 1TB SSD and then format and install Bookworm on it in the regular manner. If I do that what's the point of btrfs replace?

    Any suggestions?
    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
    – John F. Kennedy, February 26, 1962.

    Originally posted by GreyGeek View Post
    What I do not understand, nor find on the Internet, is how the sda1 and sda2 partitions get copied to sdb, because without them booting sdb would be impossible, and I'd be forced to simply replace my current SSD with the 1TB SSD and then format and install Bookworm on it in the regular manner. If I do that what's the point of btrfs replace?
    sda1 and sda2 will not get copied. They are not btrfs file systems. TL/DR: btrfs remove is not a whole-disk clone program.

    I don't think BTRFS replace - or any file system backup tool - would result automatically in a bootable drive and that wouldn't be the intent.

    If your goal is to remove sda (or just not boot from it), you would need to prepare sdb for that task - i.e. make the EFI and swap partitions, install grub, etc.

    If you goal is to move your OS from sda3 to sdb1 and still boot from sda, then replace should work just fine - I think.

    The one thing I see totally left out of that webpage AND the man page is UUID handling. No mention at all if the UUIDs are duped or replaced or what. I may have to VM this and see what happens. Clearly, the write ups are insufficient. If we assume btrfs replace duplicates the sda3 UUID on sdb1, it would also have to change (or remove) the UUID and/or file system on sda3 (or the entire device). This also assumes GRUB is booting (and your OS fstab is) using UUIDs, which is not a given. Any number of other schemes could be possible. Thus, my conclusion is the reason this is not addressed in the man page and write up is that btrfs replace is not intended - by itself - to move an entire device. Rather, it moves the file system contents from one device (partition) to another. You will still need to edit fstab and update grub before rebooting.

    So, if your goal to to replace/swap the hardware devices (not just the file system), then the steps are (as far as I can tell):
    1. Partition sba for EFI, SWAP and the btrfs file system.
    2. Format sdb1 as EFI and copy the EFI contents to it
    3. Format the SWAP on sdb2
    4. Edit fstab (still on sda3) and replace the UUIDs of EFI and SWAP with the new UUIDs on sdb1 and sdb2
    5. Run grub-install and install grub to sdb
    6. Reboot and set sdb as the boot device in your BIOS
    7. Boot up and verify you have booted from sdb EFI and sdb SWAP is in use.
    8. Run btrfs replace and transfer the contents of sda3 to sdb3
    9. Edit fstab to again to replace sda3 UUID with sdb3 UUID
    10. Reboot again and verify all is good
    11. Remove sda
    12. Expand the file system on sdb3 to use the entire partition

    Sort of an answer to your question "..what's the point of btrfs replace?" the answer is "It's a quicker and more robust (better error handling) way to move a btrfs file system from one device to another while running live."

    The other way (previously only way) was to add the second device, and then remove the previous device, and wait for the data move to end. The advantage to this method is it doesn't require the file system expansion when complete, but that only take a few seconds anyway and is totally safe.

    "btrfs replace" is (reportedly, I've not used it) faster. It would also be better than a clonzilla or DD move because btrfs will do error checking during the move and those other tools do not. I could also see a use-case where one could move to/from multi-device file systems to a single device or vice versa. Like you have 2x1TB drives using a multi-device file system and you want to move the whole thing to a single 4TB drive. No doubt that would be much faster than the "add - remove" method.

    On the other hand, clonezilla does most of the above steps for you if you're swapping out a disk. Personally, I do not like using clonezilla because it's not transparent and you don't find out something went haywire until it's too late. In my world, I would do the above steps because it's non-destructive. If something goes wrong sda is still intact.
    Last edited by oshunluvr; Oct 19, 2023, 02:07 PM. Reason: editorial and grammar

    Please Read Me


      Thank you for all that detail. On my desktop I installed a new system drive in May (2016 SATA SSD to NVMe), but proceeded unaware that btrfs replace was a thing.

      (I tested with a clean install to the new device, and then I thought only at the subvolume level, and used send/receive as seemed necessary, from read-only snapshots. This included one previous snapshot for the root and home subvolumes that I'd used for incremental backups and btrfs accepted these from the new drive; I really didn't expect that to work. I made no changes to the old SSD.)
      Regards, John Little


        oshunluvr, you pretty much detailed the conclusions I had come to, after exploring the vagaries. Rather than test out your 12-step plan for Btrfs Anonymous, I decided to go the default route and simply plug in the new SSD to my sole SATA port and install using a LiveUSB. Once it boots I'll copy my data over to the new drive and march forward from there. Or, possibly, install a fresh copy of Bookworm to the 1TB drive while it is in the plugged into my USB Hub, and then power down the system, replace my 500GB SSD with the freshly minted 1TB SSD and reboot. Should work.

        It's like the old contract -- the big print gives it to you, but the small print takes it away. I really can't see a use for btrfs replace if one can not literally move their system to a bigger drive and resize it to fill the space.

        "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
        – John F. Kennedy, February 26, 1962.


          I can see a use for btrfs replace in a server environment where down time and workload is a big deal. But not really in a home, single user environment.

          JLittle points out a salient fact that, if one uses subvolumes anyway, the usefulness of btrfs replace is lessened even further. It's almost like the developer(s) took existing functionality and bundled it into a single command for convenience's sake.

          Still, I'm curious about the actual functionality. Since I'm about to be retired (for the third time, lol) I'll have some time to play with it!

          Please Read Me


            OK testing over (not much going on today, lol).

            Test procedures:
            • Launched Ubuntu 23.04 VM with 3 virtual disks attached
              • 1 boot, 2 sacrificial (both sacrificial disks contained data from now defunct VMs)
            • Wiped disk 2 and created a new BTRFS file system
            • Copied all the data from disk 3 onto disk 2
            • Wiped disk 3
            • Mounted disk 2 at /mnt/sdb2
            • Ran "btrfs replace start 1 /dev/sdc2 /mnt/sdb2
            Here's the disk info before:
            Click image for larger version  Name:	Screenshot_20231020_100053.jpg Views:	0 Size:	62.1 KB ID:	674595

            and after:
            Click image for larger version  Name:	Screenshot_20231020_101113.jpg Views:	0 Size:	46.8 KB ID:	674596

            You can see that "btrfs replace" :
            • Moved all the data
            • Mounted the new device at the previous mount location
            • Used the same UUID
            • Removed the file system from the previous disk
            So clearly - at least to me - it DOES have a use and functionality. One thing I could have tested was the file system label, but I would bet a shiny new dollar that it would duplicate that too. So it seems, unless you're still mounting using device names, this could be a very useful tool in many circumstances, like servers.

            My media server has 20 subvolumes mounted at various locations but all on a single 16TB btrfs file system. I see that I could "replace" that drive and never have to shutdown or reboot the server at all. Cool!
            Last edited by oshunluvr; Oct 20, 2023, 08:30 AM.

            Please Read Me


              BTW, the entire operation including moving the 10GB of data took literally 10 seconds. Note that all the virtual drives are together on a NVMe v4.0 drive, so your mileage (and speed) may vary, lol.

              Please Read Me


                I wasn't brave enough to try btrfs replace, and considering what happened when I tried the old method maybe I should have.

                First, I downloaded the Debian 12.2 installer KDE iso (not Live) and burned it to a USB stick to have it ready.

                I plugged the USB ISO into the USB 3 hub and booted into it. From it I went through the install procedure, arriving at the HD formatting options. I chose manual and proceeded to create a 300MB EFI partition, a 32GB swap partition (2xRAM) and a 930GB btrfs partition, the remainder of the SSD. When I got to the final screen to commit the new partitions I noticed that it included two swap partitions, the new 32GB on the 1TB SSD, and the old 2GB on the 500GB SSD. I redid the partitioning a couple times trying to stop it from including the 2GB swap, but couldn't. It always came back.

                The result was that the new 1TB SSD, being mounted on my 5GB/s USB3 hub, ended up being linked to the installation on my old main 500GB main SSD. My laptop has only one internal drive port. When I rebooted I got the grub> screen. Using the grub commands to continue the boot I was blocked by the presence of the "@" in @rootfs. Time to do a fresh install.

                So, I pulled my 500GB drive and put the 1TB in the internal slot. It booted into the initramfs> screen, IIRC, which I have no experience working with.

                Didn't matter, I thought, because I planned to do a fresh install. After all, the Debian 12.0 Live USB went on like a silk glove, including my HP Laser Jet P1606dn, which it found and installed automatically, including the duplex. I would have to compile the rtw_88x2bu.ko for the 6.1 kernel on 12.2, but that's no chore.

                Debian 12.2 is Debian 12.0's twin sister, and I learned that when installed from and ISO and not a Live USB she is brain dead. Or at least that I my experience.

                No repositories were installed, which is the FIRST time in I had encountered that problem since repositories became the thing. That was a stunner. I had to load the sources.list manually from repository lists examples taken from the Debian's website.

                Even with repositories to work with, Discover behaved like it had a frontal lobotomy and couldn't find half the libraries and apps I needed to install. I ended up installing Synaptic because it was in the repository, but working with its search function I remembered why I switched to muon. I found a muon deb package at the KDE home webpage and DL'd and installed it. THEN, I could find what I was looking for. And, it was faster than Synaptic.

                Debian 12.2 has warts all over the place. Here are a few:
                Wayland was unstable on 12.0. It is even more so on 12.2. I switched to X11.
                All four of the openCL, GL screens on the InfoCenter -> Graphics display required installing libraries in order to display data.
                The number of little things that destroy an experience are many, but I continued on. Right now, the major problem is that my HP Laser Jet P1606dn printer won't install properly. It installs with HPLIP, but it won't print because it is missing the "proprietary" plugin, which 12.0 apparently happily supplied.
                Attempts to run fail with the "Can't find the Network connection". Copying the ppd over from a previous subvolume of 12.0 doesn't work. I've had this problem repeatedly in the last 10 years since I purchased this printer and I assign the problem entirely to HP's poor coding skills. My next printer, if I have to buy one, won't be an HP.

                Anyway, except for the printer and a few apps, my installation is running nicely.
                "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                – John F. Kennedy, February 26, 1962.