Announcement

Collapse
No announcement yet.

Snapper v Btrbk v Manual

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Snapper v Btrbk v Manual

    My plan was to analyze the source code for Btrbk but that hit a snag when I opened the source and found it to be composed of 7,345 lines of Perl. Perl has the reputation of being easy to write and hard to read. Since I have never learned, or even studied, Perl, analyzing Btrbk is way above my pay grade. From the docs, and and some code snippits it appears that Btrbk is configured to store full snapshots at
    Code:
    /mnt/btr_backup/mylaptop/rootfs.YYYYMMDD
    /mnt/btr_backup/mylaptop/home.YYYYMMDD
    which is identical to /mnt/snapshots/@yyyymmdd, and /mnt/snapshots/@homeyyyymmdd.
    So, if one wants to automate a process similar to or like the one I use then Btrbk is your tool. Also, Btrbk also has the ability to send & receive snapshots to remote locations, which Snapper cannot do, and I believe that TimeShift can't do that either. Manual commands can do it.

    I installed the version of snapper in the 20.04 repository and configured the root (/) with the default configuration.

    Here are the folders I created and used to hold snapshots:
    Code:
    :~# [B]vdir /mnt[/B] 
    total 0 
    drwxr-xr-x 1 root root 228 Feb 20 01:15 @ 
    drwxr-xr-x 1 root root 130 Mar 16 21:50 snapshots 
    
    :~# [B]vdir /mnt/snapshots[/B] 
    total 0 
    drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103122130 
    drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103132115 
    drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103142159 
    drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103152204 
    drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103162149
    Now I view the folders created by Snapper:
    Code:
    :~# [B]vdir /.snapshots/ [/B]
    total 0 
    drwxr-xr-x 1 root root 32 Mar 16 22:00 1 
    drwxr-xr-x 1 root root 32 Mar 16 22:02 3 
    drwxr-xr-x 1 root root 60 Mar 16 22:05 4 
    drwxr-xr-x 1 root root 32 Mar 17 11:05 5
    As can be seen, I've created 4 snapshots. The 2nd is mmissing because I deleted it.

    What's under snapshot 3?
    Code:
    :~#[B] vdir /.snapshots/3 [/B]
    total 4 
    -rw------- 1 root root 175 Mar 16 22:02 info.xml 
    drwxr-xr-x 1 root root 228 Feb 20 01:15 [B]snapshot [/B]
    Two files, the info.xml, which contains meta data the user entered when they created the snapshot, AND, the actual snapshot itself, labled "snapshot". Here's the info.xml contents:

    Code:
    [FONT=monospace][FONT=monospace]:~# [B]cat /.snapshots/3/info.xml [/B] 
    <?xml version="1.0"?> 
    <snapshot> 
      <type>pre</type> 
      <num>3</num> 
      <date>2021-03-17 03:02:59</date> 
      <description>apt</description> 
      <cleanup>number</cleanup> 
    </snapshot> [/FONT]
    [/FONT]


    What's in snapshot?
    Code:
    :~# [B]vdir /.snapshots/3/snapshot/[/B] 
    total 28 
    drwxr-xr-x 1 root root    0 Jan  9  2020 backup 
    lrwxrwxrwx 1 root root    7 Jan  5  2020 bin -> usr/bin 
    drwxr-xr-x 1 root root  600 Mar 12 21:32 boot 
    drwxr-xr-x 1 root root    0 Jan  5  2020 cdrom 
    drwxr-xr-x 1 root root  140 Jan  3  2020 dev 
    drwxr-xr-x 1 root root 5338 Mar 16 21:52 etc 
    drwxr-xr-x 1 root root   10 Aug  6  2020 home 
    ....
    drwxr-xr-x 1 root root    0 Jan  3  2020 srv 
    drwxr-xr-x 1 root root    0 Dec  5  2019 sys 
    drwxrwxrwt 1 root root 1950 Mar 16 22:02 tmp 
    drwxr-xr-x 1 root root  126 Dec 31 13:08 usr 
    drwxr-xr-x 1 root root  120 Feb 20 01:15 var
    My entire system!
    What is the size of it?

    Code:
    [FONT=monospace]:~# [B]btrfs fi usage /.snapshots/3/snapshot[/B] 
    Overall: 
        Device size:                 465.76GiB 
        Device allocated:            134.03GiB 
        Device unallocated:          331.73GiB 
        Device missing:                  0.00B 
        Used:                        111.60GiB 
        Free (estimated):            352.90GiB      (min: 352.90GiB) 
        Data ratio:                       1.00 
        Metadata ratio:                   1.00 
        Global reserve:              206.77MiB      (used: 0.00B) 
    
    Data,single: Size:131.00GiB, Used:109.83GiB (83.84%) 
       /dev/sda1     131.00GiB 
    Metadata,single: Size:3.00GiB, Used:1.77GiB (59.12%) 
       /dev/sda1       3.00GiB 
    System,single: Size:32.00MiB, Used:16.00KiB (0.05%) 
       /dev/sda1      32.00MiB 
    Unallocated: 
       /dev/sda1     331.73GiB 
    [/FONT]
    Yup, it's the whole ball of wax! A complete snapshot.

    Knowing this, one could use the "btrfs su snapshot" to create a rw copy of snapshot 3 and name it "@".
    Where to put @? It's under the <ROOT_FS> but it cannot be accessed from within /. Remember? You cannot "cd .." from / and move to a higher directory level, which would be <ROOTT_FS>. That's why Snapper does NOT restore by renaming @ (or @home) to @old and creating a new @ from a snapshot. How does Snapper restore to a previous snapshot? By copying one or more (or all) files from a snapshot back into the system using it's "undochange" command. A "complete" restore is done by using just the "snapper undochange x..y" command. When finished the system is back at the condition it was in when it was "x". Changes will NOT include folders and files listed in filter.txt. To undo a specific file you'd use "snapper undochange x..y somefile1 somefile2 ...".

    What won't Snapper restore?
    Code:
    [FONT=monospace][FONT=monospace][FONT=monospace][COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ [B]vdir /etc/snapper/ [/B][/COLOR]
    total 0 
    drwxr-xr-x 1 root root  8 Mar 17 11:12 configs 
    drwxr-xr-x 1 root root 14 Mar 16 21:52 config-templates 
    drwxr-xr-x 1 root root 44 Mar 16 21:52 filters 
    
    [COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ [B]vdir /etc/snapper/filters/[/B] [/COLOR]
    total 12 
    -rw-r--r-- 1 root root 104 Feb  3  2020 base.txt 
    -rw-r--r-- 1 root root  37 Feb  3  2020 lvm.txt 
    -rw-r--r-- 1 root root  14 Feb  3  2020 x11.txt 
    
    [COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ [B]cat /etc/snapper/filters/base.txt [/B][/COLOR]
    /etc/adjtime 
    /etc/mtab 
    /var/lib/logrotate.status 
    /var/lib/misc/random-seed 
    /var/lib/ntp/drift/ntp.drift
    [/FONT][/FONT][/FONT]

    And there, in base.txt, is what it won't restore. xll.txt holds "/.Xuathority".

    So, say you do "sudo apt update && sudo apt full-upgrade". Snapper creates a "Pre" snapshot before apt runs "full-upgrade" and a "Post" snapshot after the upgrade is done. Say the upgrade borks your system in some way. IF the upgrade did't crash your display and you can still access the command line, you issue
    "sudo snapper undochange x..y"
    where x is the number of the Pre snapshot and y is the number of the Post snapshot. How do you get x and y?
    By doing "sudo snapper list". Then you can use the undochange command.

    There has been several times when I've inadvertantly erased a file or folder. Without Snapper's "undochange" comand how do I recover those files? By opening a Konsole, sudo'ing to root, and mounting my drive to /mnt. Then I fire mc and put the most recent snapshot in the left panel and my system in the right panel. I highlight the files/folders I want to restore and copy them from the left panel to the right panel. To roll back the entire system I mv @ to @old and create a rw @ from the most recent snapshot. Then I reboot.
    Back in the system I delete @old.

    Snapper does NOT backup your system. Neither does making snapshots the way I do. Your system is backed up ONLY when you use the btrfs send & receive commands to move a ro snapshot to a remote drive.

    Snapper has a "diff" command. With all of its snapshots residing under / one can use Kdiff3 to view the difference between a folder in a snapshot and the same folder in another snapshot or in your system.
    The btrfs send command also has that same ability, but only for entire snapshots.

    btrfs send --no-data -p /mnt/snapshots/@202102182239 /mnt/snapshots/@202102192115 > diff.txt

    Browse diff.txt to see the differences.

    Unlike TimeShift, Snapper does NOT link your <ROOT_FS> to a subdirectory under /, which is dangerous.

    Last edited by GreyGeek; Mar 17, 2021, 02:22 PM.
    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
    – John F. Kennedy, February 26, 1962.

    #2
    I know you have mentioned this elsewhere and it seems intuitively right, but can you elaborate on why keeping the <root_fs> permanently mounted somewhere under "/" is dangerous?

    (I am wondering if I need to move away from Timeshift, but so far it is quite convenient ...)

    Comment


      #3
      Code:
      <ROOT_FS> is the root of the Binary Tree
      
                                     <root_fs>
                                        /      \
                                      /         \
                                    /            \
                                  @ as /      @home
      In /etc/fstab @ is mounted to /

      When you mount the drive to /mnt then /mnt in effect becomes <ROOT_FS> and underneath it are @ and @home.

      Snapper creates snapshots under / as
      /.snapshot/1/snapshot
      /.snapshot/2/snapshot, etc...

      And, even worse, linking <ROOT_FS> to / as TImeShift does, shown below, creates a situation in which IF @ becomes corrupted and won't mount as / in /etc/fstab, then your ALL snapshots which are under / are inaccessible.
      Code:
                                     <ROOT_FS>
                                        /         \
                                      /            \
                                    /               \
                                  @ as /      @home
                                 /\
                                /  \
                              /      \
                            /         /.snapshot/1/snapshot (etc...)
                          /
                 <ROOT_FS> ink
                        /
                       /
                      @
                 (becomes an endless "mirror reflection")
      Binary trees use head and tail addresses to create links from the previous leaf and to the next leaf. Each leaf has an address. It is possible to link a lower leaf to an upper leaf, creating a loop or mirror, in effect.

      Even if one used a LiveUSB and mounted the drive to /mnt (on the LiveUSB) and could see /mnt/@ they still could not access @ because it is corrupt. How do you roll back in order to recover if your snapshots are not accessible? I ran into this situation while testing TimeShift, but it could still happen with Snapper because it, too, puts its snapshots under /.

      In 5+ years of using Btrfs I've had only 2 orr 3 situations, all of my own making, in which @ was not mountable and I had to roll back using a snapshot. It was possible to do that because with the snapshot directory directly under <ROOT_FS> I had access to the snapshots even though @ was broken.

      Keeping the <ROOT_FS> permanently mounted to /mnt exposes both @ and the contents of /mnt/snapshots to accidental damage or destruction. I call my script from a Konsole with "sudo /make_snapshots.sh". My script mounts /mnt and /backup (a different drive) and then creates the new snapshot and sends it to /backup using an incremental send.. If both tasks are completed then /mnt and /backup are unmounted. If a problem arises the drives remain mounted and I "sudo -i" to root and manually fix the problem. So far the script has not failed.

      If I want to restore a file or a folder I open a Konsole, do "sudo -i" and run mc.. Then I browse the snapshot in the left panel of mc and @ in the right panel. I copy the files or folders from the snapshot in the left panel to the same location in @ in the right panel. Then I quit mc, unmount /mnt (and /backup if necessary), and exit the root Konsole, and exit Konsole.

      Most of what Snapper does can be done manually with Btrfs command and utilities. And, doing it manually is a lot faster than doing it is Snapper. IF I want to list my snapshots it takes just a few seconds to do it. If I run "sudo snapper list" it can take 60-90 seconds.











      [/code]
      "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
      – John F. Kennedy, February 26, 1962.

      Comment


        #4
        Umm...I think we've had this discussion before, but on-disk "delta" snapshots of @ will not help you if @ is corrupted, regardless of where they are stored (it makes no difference if they are stored inside @ or elsewhere, they are worthless). To recover from file system corruption you need an external "backup" snapshot (created with btrfs-send).

        I keep bringing this up because it's dangerous to think that on-disk "delta" snapshots could somehow help with file system corruption *if* they are not stored within the snapshotted subvolume (this isn't the case).

        And like I mentioned in another thread, you can easily make a top-level @snaphots subvolume and mount that into @/.snaphots to store the snaphots outside of @ (but like above this doesn't help in case of @ corruption, they are just as worthless as the ones stored inside @).
        Last edited by kubicle; Mar 18, 2021, 03:30 AM.

        Comment


          #5
          I use Snapper. I have a few hourly snapshots and a daily snapshot that automatically happen and delete the old ones. This is better for me than manually making them because I forget to delete them and then suddenly I have huge snapshots. Either that or I just don't get round making snapshots. Snapper makes sure that snapshots actually happen and the program, with the undo changes, has already bailed me out a couple of times on mistakes I made.

          Comment


            #6
            Originally posted by Bings View Post
            I use Snapper. I have a few hourly snapshots and a daily snapshot that automatically happen and delete the old ones. This is better for me than manually making them because I forget to delete them and then suddenly I have huge snapshots. Either that or I just don't get round making snapshots. Snapper makes sure that snapshots actually happen and the program, with the undo changes, has already bailed me out a couple of times on mistakes I made.
            It's good to have a routine that automatically makes snapshots, either by Snapper or manually, which I use. The last activity I do before shutting down my computer is to store my latest snapshot (incrementally) to another medium mounted at /backup.

            You can do this with Snapper backups as well. Assume your have several snapshots, the latest being number 18.
            btrfs su snapshot -r /.snapshot/18/snapshot /.snapshot/@yyyymmddhhmm
            then
            btrfs send /.snapshots/@yyyymmddhhmm | btrfs receive /backup
            where /backup is the mounted remote medium .
            After the send&receive is complete you can
            btrfs subvol delete -c /.snapshot/@yyyymmddhhmm
            and then umount /backup after a few sync's

            I don't know what Kubicle means by "delta" snapshots, but like I stated in the OP:
            Snapper does NOT backup your system. Neither does making snapshots the way I do. Your system is backed up ONLY when you use the btrfs send & receive commands to move a ro snapshot to a remote drive.
            which is why I have several USB drives and an onboard spinner as storage mediums for snapshots in addition to what is stored under /mnt/snapshots.

            In the last 5+ years I can think of only 2 or 3 occasions where @ didn't start when my computer was turned on, and all of them were the results of my experimenting around, not because of any BTRFS failure, and restoring from the previous daily snapshot was always easy to do. When ever I do something that carries the risk of breaking my system, like a 20+ package update, I always create a snapshot first so that I can roll back if the update borks my system. I can roll back in under two minutes, including the reboot. I've seen folks on this forum spend hours and days attempting to restore what they had before their problems arose. A couple minutes vs a couple days? The choice is a no-brainer.

            Again. ... Snapper snapshots are in /.snapshot/n/snapshot and /home/.snapshot/n/snapshot. Since @ is mounted to / and the snapshots are under / then if @ is corrupted and won't boot then you have no access to any snapshots under /. To be sure, you still can mount <ROOT_FS> to /mnt and see @ from a LiveUSB, but you have no guarantee that you can navigate @ to /.snapshot/n/snapshot and use it to create a fresh @.

            Because Snapper can't or won't mount the drive to /mnt and store snapshots under <ROOT_FS>/snapshot/ its success depends entirely on @ being able to boot to enable access to its snapshots. The other negative is that Snapper doesn't do a true rollback. It does an internal diff and reverses the "+"'s, the "-"'s and the "c"'s in the current "0" snapshot, which can/should never be deleted. Despite that it doesn't "reverse" files and folders listed in filters.txt, so a true image of the previous snapshot is never restored.
            When done manually, as I've shown, the rollback IS the previous @. And, I can use "sudo mc" and, as I've written before, put the snapshot in the left panel and the live system in the right panel, and copy and folder or file in the left panel to the right panel to restore just a few components, or I can add something new in the right panel to a previous snapshot by copying it to the previous snapshot. Btrfs is sooooo flexible.
            "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
            – John F. Kennedy, February 26, 1962.

            Comment


              #7
              snapper autoremoved

              Just this morning, snapper stopped running.

              snapper was gone. "command not found: snapper" What?

              I'd been cleaning up, and had run apt autoclean, apt-get clean, and apt autoremove. Checking /var/log/apt/history.log, I found:

              Commandline: apt autoremove
              Requested-By: jlittle (1000)
              Remove: linux-image-5.8.0-43-generic:amd64 (5.8.0-43.49), linux-modules-extra-5.8.0-43-generic:amd64 (5.8.0-43.49), libbtrfs0:amd64 (5.7-1), libgtksourceview-3.0-common:amd64 (3.24.11-2), linux-headers-5.8.0-43:amd64 (5.8.0-43.49), snapper:amd64 (0.8.12-1), linux-headers-5.8.0-43-generic:amd64 (5.8.0-43.49), linux-modules-5.8.0-43-generic:amd64 (5.8.0-43.49), gir1.2-gtksource-3.0:amd64 (3.24.11-2), libsnapper5:amd64 (0.8.12-1), libgtksourceview-3.0-1:amd64 (3.24.11-2)
              End-Date: 2021-03-19 08:19:36
              Why? I've no clue.**

              Reinstalling snapper didn't work; it didn't pick up the configuration, even though the files are all there. Maybe some systemd-fu would have sorted it.

              Anyway, snapper itself provided the solution, I snapshotted the automatically generated snapshot to the top level, edited its fstab, and dropped a custom.cfg into /boot/grub that boots my new snapshot, and rebooted. Snapper is running normally now.

              I'm concerned that this happened silently, except for a couple of words I needed to see in ~500 characters of dense version number stuff. For an entirely unrelated reason, a failed Windows update overnight, I was reviewing space and backups, hence the clean-up.

              ** I have a clue now, after reviewing my command log. I'd got snapper as a dependency of snapper-gui, which had a problem as packaged in Ubuntu. There's a fix on the github repository, so I'd cloned and built it, after uninstalling the packaged version. A trap for the unwary.

              I've now used apt to get "snapper set to manually installed."
              Regards, John Little

              Comment


                #8
                Originally posted by GreyGeek View Post
                Again. ... Snapper snapshots are in /.snapshot/n/snapshot and /home/.snapshot/n/snapshot. Since @ is mounted to / and the snapshots are under / then if @ is corrupted and won't boot then you have no access to any snapshots under /. To be sure, you still can mount <ROOT_FS> to /mnt and see @ from a LiveUSB, but you have no guarantee that you can navigate @ to /.snapshot/n/snapshot and use it to create a fresh @.
                You keep saying that, but that is a completely moot point. If you have file system corruption no on-disk btrfs snapshot will help you recover from it whether you have "easy" access to it or not. It does not matter if the snapshot is stored within @ or outside of it. Btrfs snapshots don't protect against file system corruption, period. Only backups will. Of course there are many reasons why @ wouldn't boot (of which file system corruption is only one...like a corrupted file, grub problem, fstab errors etc.).

                Your snapshots are either usable or not in recovering from the issue. If the issue is recoverable with snapshots that are not stored within @, it's also recoverable with snapshots stored within @ (by mounting with a liveISO, for example). If the issue is catastrophic file system corruption, no snapshot will be usable regardless of where it resides on-disk.

                And again...You don't have to have the snapper snapshots stored within @ if you don't wish to, you can have a separate @snaphots subvolume and mount that into @/.snapshots (in which case snapper stores the snapshots in @snapshots and not @), but this makes absolutely no difference in case of file system corruption).

                Originally posted by GreyGeek View Post
                Because Snapper can't or won't mount the drive to /mnt and store snapshots under <ROOT_FS>/snapshot/ its success depends entirely on @ being able to boot to enable access to its snapshots.
                You don't have to be able to boot to @ to access the snapper snaphots. You do not even need to mount (or be able to mount) @.
                Example: the below shows a nested subvolume structure (not done with snapper, but similar):
                ID 257 gen 19 top level 5 path @
                ID 258 gen 12 top level 5 path @home
                ID 259 gen 16 top level 257 path <FS_TREE>/@/.snapshots/snapshot_of_@

                The ID 259 subvolume is a snapshot of @ that is in
                @/.snapshots/ (note the "top level 257" that means it's nested inside ID 257=@ [using the term "inside" loosely here, as it's not actually physically inside @])
                I can easily mount that with, for example: "sudo mount -o subvolid=259 /dev/sdb1 /mnt" (no need to mount @ first [or at all], you can use the snapshot directly even if it is nested "inside" @ , and you can also do everything that you could do with it if it was not "inside" @...there is no distinction. snapshots are always individual subvolumes even if they are nested "inside" other subvolumes [like they are if you are using snapper] and you can use them the same way...you can replace @ with it, boot into it, snapshot it, back it up etc.)

                ---

                All the above doesn't mean one or the other (having a flat or nested subvolume structure in a btrfs file system) won't be simpler or easier (like requiring fewer steps), but this of course depends on the issue (and to a point, your preferences). There are pros and cons to either approach (but there really are no "showstopper" issues that would make either preferable in every use case). Your logical chain is "I like to have a flat snapshot subvolume structure, snapper uses a nested subvolume structure, snapper is faulty", but the issues with a nested structure you are describing are not based in reality, and not how nested btrfs snaphots actually work.
                Last edited by kubicle; Mar 19, 2021, 08:45 AM.

                Comment


                  #9
                  To do ""sudo mount -o subvolid=259 /dev/sdb1 /mnt" " you have to boot some platform, and one with Btrfs loaded or loadable. Hence the LiveUSB or whatever. My own experience is that if @ is corrupted and won't boot then nothing inside of @ is accessible either. I could not browse @ after mounting my /dev/sda1 to /mnt, but I could access /mnt/snapshots/@yyyymmdd and create a new @ from it. As you said in #4 "you can easily make a top-level @snaphots subvolume and mount that into @/.snaphots to store the snaphots outside of @ (but like above this doesn't help in case of @ corruption, they are just as worthless as the ones stored inside @)" That's what I've been saying.
                  "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                  – John F. Kennedy, February 26, 1962.

                  Comment


                    #10
                    Originally posted by GreyGeek View Post
                    then nothing inside of @ is accessible either.
                    But the snapper snapshots at @/.snapshots/ are not actually "inside" @ (it only looks that way on the surface, because they are nested), they are separate subvolumes that can be mounted separately/independently from @ (although, due to them being nested, they do also get automatically mounted if you do mount @...which can obviously confuse one to think that they are inside @). You do not need the ability to mount @ and/or browse it.

                    I could not browse @ after mounting my /dev/sda1 to /mnt, but I could access /mnt/snapshots/@yyyymmdd and create a new @ from it.
                    Then you could also have mounted your snapshot @yyyymmdd even if it had been nested within @ (like in @/path/to/whatever), and also replace @ with it. There exist no issues that you could have on @ that would only affect one type of snapshot and not the other (nested/non-nested).

                    The trick with the above is not to mount /dev/sda1 and try to browse @, but to mount the nested snapshot subvolume directly (if you wish to browse the contents of it), or you can simply mount /dev/sda1 and then use the btrfs command rundown to replace @ with the nested snapshot...in a mostly similar way you probably did with your non-nested snapshot).

                    Btrfs developers are reasonably smart people. If there were issues with nested subvolume becoming inaccesible if the parent is inaccessible while non-nested subvolumes would still function, they wouldn't have made nesting subvolumes possible (the snapper devs wouldn't have been using nested snapshots...and we wouldn't be having this discussion). There are file system issues that do make snapshots inaccessible or unusable, but these issues affect both types in a similar way.
                    Last edited by kubicle; Mar 19, 2021, 12:56 PM.

                    Comment


                      #11
                      I can access Snapper's root snapshots as root at /.snapshot without mounting the <ROOT_FS> (the HD or SSD) to /mnt. I can also access @home's snapshots at /home/.snapshot without mounting <ROOT_FS> As root I can cd to / and see /.snapshot.

                      However, I cannot "cd .." upward to the level of <ROOT_FS>, hence <ROOT_FS>, @ and @home are not accessible to root without mounting <ROOT_FS>.

                      IMO, snapper's snapshots are definitely INSIDE @, which is mounted as / in fstab.

                      Btrfs gives every subvolume an ID, and a Btrfs command even shows what cannot be accessed even as root unless one mounts <ROOT_FS>
                      Code:
                      [FONT=monospace][COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ [B]sudo btrfs subvol show / [/B][/COLOR]
                      @ 
                              Name:                   @ 
                              UUID:                   e11be9cf-95ff-f640-834e-0ed05439b51b 
                              Parent UUID:            d2001490-2064-2a4c-a870-1be84141a78a 
                              Received UUID:          - 
                              Creation time:          2021-03-15 14:43:28 -0500 
                              Subvolume ID:           1695 
                              Generation:             969955 
                              Gen at creation:        965705 
                              [B]Parent ID:              5[/B] 
                              Top level ID:           5 
                              Flags:                  - 
                              Snapshot(s): 
                                                      snapshots/@202103152204 
                                                      snapshots/@202103162149 
                                                      snapshots/@202103171903 
                                                      snapshots/@202103182218 
                              Quota group:            0/1695 
                                Limit referenced:     - 
                                Limit exclusive:      - 
                                Usage referenced:     102.32GiB 
                                Usage exclusive:      241.61MiB[/FONT]
                      When I do "sudo btrfs subvol list /"
                      Code:
                      [FONT=monospace][B][COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ sudo btrfs subvol list / [/COLOR][/B]
                      ID 1694 gen 965705 top level 5 path snapshots/@202103142159 
                      ID 1695 gen 969969 top level 5 path @ 
                      ID 1696 gen 966534 top level 5 path snapshots/@202103152204 
                      ID 1697 gen 967268 top level 5 path snapshots/@202103162149 
                      ID 1724 gen 968231 top level 5 path snapshots/@202103171903 
                      ID 1725 gen 969768 top level 5 path snapshots/@202103182218[/FONT]
                      All of my snapshots also have an ID, but that doesn't mean that they are under @. It means that they are under the <ROOT_FS>. My @ ID is 1695. The "Parent ID" of @ is 5, which is <ROOT_FS>. @ is a child of the Top Level ID 5, which is <ROOT_FS>, i.e., the base subvolume formatted onto the disk when Ubuntu was installed with Btrfs as the root fs, and onto which every thing else is written.

                      My snapshots do NOT have "@" prefixed in front of their paths.
                      Snapper snapshots show the path "/.snapshot/n/, which contains info.xml and snapshot, the actual suvolume.

                      IMO, snapper snapshots ARE inside @, or one would have to mount <ROOT_FS> to see them. It may arise that @ is not mountable but can still be browsed. If that happens then it may be possible to create a new @ from /.snapshot/n/snapshot and boot into it, which is a "rollback" method that Snapper does not do nor does it support, because one would have to boot into a LiveUSB, mount the HD to /mnt as root and do the work manually.

                      I don't have any @ snapshots inside my / hierarchy so when @ doesn't mount, which has happened only 3 times in the last 5+ years, I go the LiveUSB route and restore @ from a snapshot that is obviously not under @. For problems not arising from a busted @ subvolume I mount my SSD to /mnt as <ROOT_FS> and do the manual commands necessary to roll @ back to a snapshot I created just before I created the problem or the updates failed. Snapshots take about a minute, including incremental backups to my remote medium, and a rollback takes about 2 minutes. A minute later, after the reboot, I am working on my desktop again.
                      "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                      – John F. Kennedy, February 26, 1962.

                      Comment


                        #12
                        Originally posted by GreyGeek View Post
                        I can access Snapper's root snapshots as root at /.snapshot without mounting the <ROOT_FS> (the HD or SSD) to /mnt. I can also access @home's snapshots at /home/.snapshot without mounting <ROOT_FS> As root I can cd to / and see /.snapshot.
                        Yes you can, because nested snapshots do get mounted automatically if you mount the "parent" subvolume (this is one of the benefits of having a nested structure, as I mentioned there are pros and cons to either structure). In case of snapper, when @ is mounted, the nested snapshot subvolumes get mounted to @/.snapshots/ automatically (the same goes for @home and it's nested snapshots, that are mounted to @home/.snapshots/). You don't need to mount them separately from @ (but you can if you need to or want to).

                        This is conceptually the same than what happens if you mount the <ROOT_FS>, it's nested "child" subvolumes ( for example @ and @home), appear automatically into the mountpoint, but they can (and usually are) also be mounted separately like mounting @ to / and @home to /home...but you can also mount @ to /mnt or /path/to/wherever...you can do the same with nested snapshot subvolumes independently of their "parent". This also explains why you can't "cd" to the parent from the directory you have mounted a nested subvolume in, as this "hides" the parent (if you mount @ at /, you can't "cd" from / to <ROOT_FS> [the parent of @], similar thing happens if you mount a nested snapshot somewhere)...this does not mean you can't do all the btrfs magic with the nested subvolume like replacing @ with a nested snapshot of @ (this process goes quite the same as with a non-nested snapshot, with mainly subtle differences))

                        .
                        Originally posted by GreyGeek View Post
                        IMO, snapper's snapshots are definitely INSIDE @, which is mounted as / in fstab
                        This means you haven't fully grasped the concept of nested subvolumes. They are not inside @, it only seems that way.
                        I can say I'm absolutely sure I'm correct (this is not an opinion), but you don't have to take my word for it:

                        Subvolumes can be nested and each subvolume (except the top-level subvolume) has a parent subvolume. Mounting a subvolume also makes any of its nested child subvolumes available at their respective location relative to the mount-point.
                        (nested) Subvolumes don't need to be mounted manually (or via fstab) to their desired locations, they "appear automatically" at their respective locations.
                        (quotes above from: https://btrfs.wiki.kernel.org/index.php/SysadminGuide)
                        This is a good read and explains most of the concepts quite well.

                        Originally posted by GreyGeek View Post
                        IMO, snapper snapshots ARE inside @, or one would have to mount <ROOT_FS> to see them.
                        This isn't correct, I'm not voicing my opinions, only what can actually be read in btrfs documentation.

                        Originally posted by GreyGeek View Post
                        I don't have any @ snapshots inside my / hierarchy so when @ doesn't mount, which has happened only 3 times in the last 5+ years, I go the LiveUSB route and restore @ from a snapshot that is obviously not under @. For problems not arising from a busted @ subvolume I mount my SSD to /mnt as <ROOT_FS> and do the manual commands necessary to roll @ back to a snapshot I created just before I created the problem or the updates failed. Snapshots take about a minute, including incremental backups to my remote medium, and a rollback takes about 2 minutes. A minute later, after the reboot, I am working on my desktop again.
                        I do not doubt that at all, nor am I suggesting that nested snapshots are somehow better than what you are doing (I quite prefer having "non-nested" snapshot structure myself, and I'm not alone in that). I'm just telling you that all that can be also be done with nested snapshots (like the ones used by snapper)...and trying to correct the misconceptions you're voicing about nested snapshots, like the idea that it is possible to have an issue with @ that is recoverable with a non-nested snapshot of @, but not recoverable with a nested snapshot of @...or that you need to be able to boot into @ or mount @ to access snapper snapshots.
                        Last edited by kubicle; Mar 19, 2021, 05:33 PM.

                        Comment


                          #13
                          Originally posted by GreyGeek View Post
                          When I do "sudo btrfs subvol list /"
                          Code:
                          [FONT=monospace][B][COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ sudo btrfs subvol list / [/COLOR][/B]
                          ID 1694 gen 965705 top level 5 path snapshots/@202103142159 
                          ID 1695 gen 969969 top level 5 path @ 
                          ID 1696 gen 966534 top level 5 path snapshots/@202103152204 
                          ID 1697 gen 967268 top level 5 path snapshots/@202103162149 
                          ID 1724 gen 968231 top level 5 path snapshots/@202103171903 
                          ID 1725 gen 969768 top level 5 path snapshots/@202103182218[/FONT]
                          All of my snapshots also have an ID, but that doesn't mean that they are under @. It means that they are under the <ROOT_FS>. My @ ID is 1695. The "Parent ID" of @ is 5, which is <ROOT_FS>. @ is a child of the Top Level ID 5, which is <ROOT_FS>, i.e., the base subvolume formatted onto the disk when Ubuntu was installed with Btrfs as the root fs, and onto which every thing else is written.

                          My snapshots do NOT have "@" prefixed in front of their paths.
                          Snapper snapshots show the path "/.snapshot/n/, which contains info.xml and snapshot, the actual suvolume.
                          Sorry for the delayed response to this one, I had to actually install snapper (and create a single snapshot with it) so I could show you the structure of snapper's nested structure:
                          (this is a btrfs partition mounted into /mnt after making a snaphot)
                          ID 257 gen 630120 top level 5 path @
                          ID 297 gen 630117 top level 257 path @/.snapshots
                          ID 298 gen 630116 top level 297 path @/.snapshots/1/snapshot

                          both the @/
                          .snapshots (ID 297) and the @/.snapshots/1/snapshot (ID 298) are individual subvolumes (that can be mounted/worked with independently from @, like all subvolumes can be). But, since both are nested [297 is a child of 257=(@) and 298 is a child of 297] they get automatically mounted as /.snapshots and /.snapshots/1/snapshot if @ is mounted to /. This does *not* mean they are inside @ (they aren't), they are separate subvolumes (and can be worked with like separate subvolumes in all practical and theoretical purposes one would associate with non-nested subvolumes, they only appear to be inside @ if you mount @ (because they are nested subvolumes).

                          In case I haven't worded it strongly enough previously, it's utterly impossible to create snapshots (or any subvolumes) that are actually inside another subvolume with btrfs, it simply cannot be done. Not by snapper, not manually. All snaphots are individual subvolumes.

                          For other comments, see my previous post.

                          Last edited by kubicle; Mar 19, 2021, 06:34 PM.

                          Comment


                            #14
                            I see my problem: I was confusing Nested subvolumes with Flat subvolumes.
                            https://btrfs.wiki.kernel.org/index.php/SysadminGuide

                            Flat (which is what I use)
                            Flat

                            Subvolumes are children of the top level subvolume (ID 5), typically directly below in the hierarchy or below some directories belonging to the top level subvolume, but especially not nested below other subvolumes,...
                            With one of the advantages being:
                            Everything in the volume that's not beneath a subvolume that has been mounted, is not accessible or even visible. This may be beneficial for security, especially when being used with snapshots, see below.

                            Nested (which I've never knew about, or if I ever did I'd forgotten about it)
                            Nested

                            Subvolumes are located anywhere in the file hierarchy, typically at their desired locations ..., especially below other subvolumes that are not the top-level subvolume
                            Its advantages and disadvantages are
                            This has several implications (advantages and disadvantages):

                            • Subvolumes don't need to be mounted manually (or via fstab) to their desired locations, they "appear automatically" at their respective locations.
                            • For each of these subvolumes the mount options of their mount point applies.
                            • Everything is visible. ...
                            • Nested subvolumes are not going to be part of snapshots created from their parent subvolume.
                            That last disadvantage is what I misinterpreted as meaning that subvolumes can't be nested. In a normal install two subvolumes are created under the <ROOT-FS> volume: @ and @home. They are mounted separately in fstab. If one navigates @ they will see the /home folder but will notice it is empty. The /home folder in snapshots of @ will be empty as well, requiring the user to snapshot both @ and @home to create a complete archive. That I've known from the beginning of my usage of Btrfs and assumed that snapshots can't be nested unless a normal directory is between them, such as /n/ in the snapshot situation.

                            With Snapper the root snapshot hierarchy under / is
                            /.snapshot/
                            which contains a normal folder with a numeric label n.
                            /.snapshot/n/
                            which contains
                            info.xml The meta data for the n'th snapper snapshot
                            snapshot The actual snapshot of @.

                            I don't have a @home subvolume.
                            Sorry for the mixup, Kubicle. (a.k.a. E.F. Hutton!)
                            "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                            – John F. Kennedy, February 26, 1962.

                            Comment


                              #15
                              Absolutely no harm done. One of the pros of a flat structure that it is generally easier to grasp a hold of (both conceptually and procedurally). Especially for people who have spent years working with earlier file systems and their partitioning schemes (flat structure shares some resemblance to that, while nested can feel somewhat alien...at least at first). Of course, snapper does it best to automate the restoration process so one could avoid the pitfalls (nested subvolume can be trickier to work with manually, even though it's possible to do the same things manually as with a flat structure).

                              Originally posted by GreyGeek View Post
                              • Nested subvolumes are not going to be part of snapshots created from their parent subvolume.
                              This is a necessary "restriction". If snapshots of a subvolume would include nested "child" subvolumes, you'd see the endless recursion of snapshot_of_@>>@>>snapshot_of_@>>@>>snapshot_of_@> >ad_infinitum in snapshots (that I think you referred to earlier in the thread). Because of this "restriction", that doesn't happen (note that the subvolume separation is also true with a flat structure subvolumes, if you take a snapshot of @, it does not include @home because they are separate subvolumes, just like nested subvolumes are).

                              This feature can also be a pro, for example, in the regard that by clever planning, you can easily exclude certain parts of the filesystem (that don't need to be snapshotted) from snapshots if you make those parts nested subvolumes (to save space,disk I/O etc.).

                              But like I said the pros and cons of flat, nested or mixed structure greatly depends on your needs and preferences (it's fairly difficult to give general recommendations that would fit all).
                              Last edited by kubicle; Mar 19, 2021, 07:30 PM.

                              Comment

                              Working...
                              X