Snapper v Btrbk v Manual

[kubicle]

Again. ... Snapper snapshots are in /.snapshot/n/snapshot and /home/.snapshot/n/snapshot. Since @ is mounted to / and the snapshots are under / then if @ is corrupted and won't boot then you have no access to any snapshots under /. To be sure, you still can mount <ROOT_FS> to /mnt and see @ from a LiveUSB, but you have no guarantee that you can navigate @ to /.snapshot/n/snapshot and use it to create a fresh @.

You keep saying that, but that is a completely moot point. If you have file system corruption no on-disk btrfs snapshot will help you recover from it whether you have "easy" access to it or not. It does not matter if the snapshot is stored within @ or outside of it. Btrfs snapshots don't protect against file system corruption, period. Only backups will. Of course there are many reasons why @ wouldn't boot (of which file system corruption is only one...like a corrupted file, grub problem, fstab errors etc.).

Your snapshots are either usable or not in recovering from the issue. If the issue is recoverable with snapshots that are not stored within @, it's also recoverable with snapshots stored within @ (by mounting with a liveISO, for example). If the issue is catastrophic file system corruption, no snapshot will be usable regardless of where it resides on-disk.

And again...You don't have to have the snapper snapshots stored within @ if you don't wish to, you can have a separate @snaphots subvolume and mount that into @/.snapshots (in which case snapper stores the snapshots in @snapshots and not @), but this makes absolutely no difference in case of file system corruption).

Because Snapper can't or won't mount the drive to /mnt and store snapshots under <ROOT_FS>/snapshot/ its success depends entirely on @ being able to boot to enable access to its snapshots.

You don't have to be able to boot to @ to access the snapper snaphots. You do not even need to mount (or be able to mount) @.
Example: the below shows a nested subvolume structure (not done with snapper, but similar):
ID 257 gen 19 top level 5 path @
ID 258 gen 12 top level 5 path @home
ID 259 gen 16 top level 257 path <FS_TREE>/@/.snapshots/snapshot_of_@

The ID 259 subvolume is a snapshot of @ that is in @/.snapshots/ (note the "top level 257" that means it's nested inside ID 257=@ [using the term "inside" loosely here, as it's not actually physically inside @])
I can easily mount that with, for example: "sudo mount -o subvolid=259 /dev/sdb1 /mnt" (no need to mount @ first [or at all], you can use the snapshot directly even if it is nested "inside" @ , and you can also do everything that you could do with it if it was not "inside" @...there is no distinction. snapshots are always individual subvolumes even if they are nested "inside" other subvolumes [like they are if you are using snapper] and you can use them the same way...you can replace @ with it, boot into it, snapshot it, back it up etc.)

---

All the above doesn't mean one or the other (having a flat or nested subvolume structure in a btrfs file system) won't be simpler or easier (like requiring fewer steps), but this of course depends on the issue (and to a point, your preferences). There are pros and cons to either approach (but there really are no "showstopper" issues that would make either preferable in every use case). Your logical chain is "I like to have a flat snapshot subvolume structure, snapper uses a nested subvolume structure, snapper is faulty", but the issues with a nested structure you are describing are not based in reality, and not how nested btrfs snaphots actually work.

[jlittle]

snapper autoremoved

Just this morning, snapper stopped running.

snapper was gone. "command not found: snapper" What?

I'd been cleaning up, and had run apt autoclean, apt-get clean, and apt autoremove. Checking /var/log/apt/history.log, I found:

Commandline: apt autoremove
Requested-By: jlittle (1000)
Remove: linux-image-5.8.0-43-generic:amd64 (5.8.0-43.49), linux-modules-extra-5.8.0-43-generic:amd64 (5.8.0-43.49), libbtrfs0:amd64 (5.7-1), libgtksourceview-3.0-common:amd64 (3.24.11-2), linux-headers-5.8.0-43:amd64 (5.8.0-43.49), snapper:amd64 (0.8.12-1), linux-headers-5.8.0-43-generic:amd64 (5.8.0-43.49), linux-modules-5.8.0-43-generic:amd64 (5.8.0-43.49), gir1.2-gtksource-3.0:amd64 (3.24.11-2), libsnapper5:amd64 (0.8.12-1), libgtksourceview-3.0-1:amd64 (3.24.11-2)
End-Date: 2021-03-19 08:19:36

Why? I've no clue.**

Reinstalling snapper didn't work; it didn't pick up the configuration, even though the files are all there. Maybe some systemd-fu would have sorted it.

Anyway, snapper itself provided the solution, I snapshotted the automatically generated snapshot to the top level, edited its fstab, and dropped a custom.cfg into /boot/grub that boots my new snapshot, and rebooted. Snapper is running normally now.

I'm concerned that this happened silently, except for a couple of words I needed to see in ~500 characters of dense version number stuff. For an entirely unrelated reason, a failed Windows update overnight, I was reviewing space and backups, hence the clean-up.

** I have a clue now, after reviewing my command log. I'd got snapper as a dependency of snapper-gui, which had a problem as packaged in Ubuntu. There's a fix on the github repository, so I'd cloned and built it, after uninstalling the packaged version. A trap for the unwary.

I've now used apt to get "snapper set to manually installed."

[GreyGeek]

I use Snapper. I have a few hourly snapshots and a daily snapshot that automatically happen and delete the old ones. This is better for me than manually making them because I forget to delete them and then suddenly I have huge snapshots. Either that or I just don't get round making snapshots. Snapper makes sure that snapshots actually happen and the program, with the undo changes, has already bailed me out a couple of times on mistakes I made.

It's good to have a routine that automatically makes snapshots, either by Snapper or manually, which I use. The last activity I do before shutting down my computer is to store my latest snapshot (incrementally) to another medium mounted at /backup.

You can do this with Snapper backups as well. Assume your have several snapshots, the latest being number 18.
btrfs su snapshot -r /.snapshot/18/snapshot /.snapshot/@yyyymmddhhmm
then
btrfs send /.snapshots/@yyyymmddhhmm | btrfs receive /backup
where /backup is the mounted remote medium .
After the send&receive is complete you can
btrfs subvol delete -c /.snapshot/@yyyymmddhhmm
and then umount /backup after a few sync's

I don't know what Kubicle means by "delta" snapshots, but like I stated in the OP:

Snapper does NOT backup your system. Neither does making snapshots the way I do. Your system is backed up ONLY when you use the btrfs send & receive commands to move a ro snapshot to a remote drive.

which is why I have several USB drives and an onboard spinner as storage mediums for snapshots in addition to what is stored under /mnt/snapshots.

In the last 5+ years I can think of only 2 or 3 occasions where @ didn't start when my computer was turned on, and all of them were the results of my experimenting around, not because of any BTRFS failure, and restoring from the previous daily snapshot was always easy to do. When ever I do something that carries the risk of breaking my system, like a 20+ package update, I always create a snapshot first so that I can roll back if the update borks my system. I can roll back in under two minutes, including the reboot. I've seen folks on this forum spend hours and days attempting to restore what they had before their problems arose. A couple minutes vs a couple days? The choice is a no-brainer.

Again. ... Snapper snapshots are in /.snapshot/n/snapshot and /home/.snapshot/n/snapshot. Since @ is mounted to / and the snapshots are under / then if @ is corrupted and won't boot then you have no access to any snapshots under /. To be sure, you still can mount <ROOT_FS> to /mnt and see @ from a LiveUSB, but you have no guarantee that you can navigate @ to /.snapshot/n/snapshot and use it to create a fresh @.

Because Snapper can't or won't mount the drive to /mnt and store snapshots under <ROOT_FS>/snapshot/ its success depends entirely on @ being able to boot to enable access to its snapshots. The other negative is that Snapper doesn't do a true rollback. It does an internal diff and reverses the "+"'s, the "-"'s and the "c"'s in the current "0" snapshot, which can/should never be deleted. Despite that it doesn't "reverse" files and folders listed in filters.txt, so a true image of the previous snapshot is never restored.
When done manually, as I've shown, the rollback IS the previous @. And, I can use "sudo mc" and, as I've written before, put the snapshot in the left panel and the live system in the right panel, and copy and folder or file in the left panel to the right panel to restore just a few components, or I can add something new in the right panel to a previous snapshot by copying it to the previous snapshot. Btrfs is sooooo flexible.

[Bings]

I use Snapper. I have a few hourly snapshots and a daily snapshot that automatically happen and delete the old ones. This is better for me than manually making them because I forget to delete them and then suddenly I have huge snapshots. Either that or I just don't get round making snapshots. Snapper makes sure that snapshots actually happen and the program, with the undo changes, has already bailed me out a couple of times on mistakes I made.

[kubicle]

Umm...I think we've had this discussion before, but on-disk "delta" snapshots of @ will not help you if @ is corrupted, regardless of where they are stored (it makes no difference if they are stored inside @ or elsewhere, they are worthless). To recover from file system corruption you need an external "backup" snapshot (created with btrfs-send).

I keep bringing this up because it's dangerous to think that on-disk "delta" snapshots could somehow help with file system corruption *if* they are not stored within the snapshotted subvolume (this isn't the case).

And like I mentioned in another thread, you can easily make a top-level @snaphots subvolume and mount that into @/.snaphots to store the snaphots outside of @ (but like above this doesn't help in case of @ corruption, they are just as worthless as the ones stored inside @).

[GreyGeek]

<ROOT_FS> is the root of the Binary Tree

                               <root_fs>
                                  /      \
                                /         \
                              /            \
                            @ as /      @home

In /etc/fstab @ is mounted to /

When you mount the drive to /mnt then /mnt in effect becomes <ROOT_FS> and underneath it are @ and @home.

Snapper creates snapshots under / as
/.snapshot/1/snapshot
/.snapshot/2/snapshot, etc...

And, even worse, linking <ROOT_FS> to / as TImeShift does, shown below, creates a situation in which IF @ becomes corrupted and won't mount as / in /etc/fstab, then your ALL snapshots which are under / are inaccessible.

                               <ROOT_FS>
                                  /         \
                                /            \
                              /               \
                            @ as /      @home
                           /\
                          /  \
                        /      \
                      /         /.snapshot/1/snapshot (etc...)
                    /
           <ROOT_FS> ink
                  /
                 /
                @
           (becomes an endless "mirror reflection")

Binary trees use head and tail addresses to create links from the previous leaf and to the next leaf. Each leaf has an address. It is possible to link a lower leaf to an upper leaf, creating a loop or mirror, in effect.

Even if one used a LiveUSB and mounted the drive to /mnt (on the LiveUSB) and could see /mnt/@ they still could not access @ because it is corrupt. How do you roll back in order to recover if your snapshots are not accessible? I ran into this situation while testing TimeShift, but it could still happen with Snapper because it, too, puts its snapshots under /.

In 5+ years of using Btrfs I've had only 2 orr 3 situations, all of my own making, in which @ was not mountable and I had to roll back using a snapshot. It was possible to do that because with the snapshot directory directly under <ROOT_FS> I had access to the snapshots even though @ was broken.

Keeping the <ROOT_FS> permanently mounted to /mnt exposes both @ and the contents of /mnt/snapshots to accidental damage or destruction. I call my script from a Konsole with "sudo /make_snapshots.sh". My script mounts /mnt and /backup (a different drive) and then creates the new snapshot and sends it to /backup using an incremental send.. If both tasks are completed then /mnt and /backup are unmounted. If a problem arises the drives remain mounted and I "sudo -i" to root and manually fix the problem. So far the script has not failed.

If I want to restore a file or a folder I open a Konsole, do "sudo -i" and run mc.. Then I browse the snapshot in the left panel of mc and @ in the right panel. I copy the files or folders from the snapshot in the left panel to the same location in @ in the right panel. Then I quit mc, unmount /mnt (and /backup if necessary), and exit the root Konsole, and exit Konsole.

Most of what Snapper does can be done manually with Btrfs command and utilities. And, doing it manually is a lot faster than doing it is Snapper. IF I want to list my snapshots it takes just a few seconds to do it. If I run "sudo snapper list" it can take 60-90 seconds.











[/code]

[Chopstick]

I know you have mentioned this elsewhere and it seems intuitively right, but can you elaborate on why keeping the <root_fs> permanently mounted somewhere under "/" is dangerous?

(I am wondering if I need to move away from Timeshift, but so far it is quite convenient ...)

[GreyGeek]

My plan was to analyze the source code for Btrbk but that hit a snag when I opened the source and found it to be composed of 7,345 lines of Perl. Perl has the reputation of being easy to write and hard to read. Since I have never learned, or even studied, Perl, analyzing Btrbk is way above my pay grade. From the docs, and and some code snippits it appears that Btrbk is configured to store full snapshots at

/mnt/btr_backup/mylaptop/rootfs.YYYYMMDD
/mnt/btr_backup/mylaptop/home.YYYYMMDD

which is identical to /mnt/snapshots/@yyyymmdd, and /mnt/snapshots/@homeyyyymmdd.
So, if one wants to automate a process similar to or like the one I use then Btrbk is your tool. Also, Btrbk also has the ability to send & receive snapshots to remote locations, which Snapper cannot do, and I believe that TimeShift can't do that either. Manual commands can do it.

I installed the version of snapper in the 20.04 repository and configured the root (/) with the default configuration.

Here are the folders I created and used to hold snapshots:

:~# [B]vdir /mnt[/B] 
total 0 
drwxr-xr-x 1 root root 228 Feb 20 01:15 @ 
drwxr-xr-x 1 root root 130 Mar 16 21:50 snapshots 

:~# [B]vdir /mnt/snapshots[/B] 
total 0 
drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103122130 
drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103132115 
drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103142159 
drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103152204 
drwxr-xr-x 1 root root 208 Feb 20 01:15 @202103162149

Now I view the folders created by Snapper:

:~# [B]vdir /.snapshots/ [/B]
total 0 
drwxr-xr-x 1 root root 32 Mar 16 22:00 1 
drwxr-xr-x 1 root root 32 Mar 16 22:02 3 
drwxr-xr-x 1 root root 60 Mar 16 22:05 4 
drwxr-xr-x 1 root root 32 Mar 17 11:05 5

As can be seen, I've created 4 snapshots. The 2nd is mmissing because I deleted it.

What's under snapshot 3?

:~#[B] vdir /.snapshots/3 [/B]
total 4 
-rw------- 1 root root 175 Mar 16 22:02 info.xml 
drwxr-xr-x 1 root root 228 Feb 20 01:15 [B]snapshot [/B]

Two files, the info.xml, which contains meta data the user entered when they created the snapshot, AND, the actual snapshot itself, labled "snapshot". Here's the info.xml contents:

[FONT=monospace][FONT=monospace]:~# [B]cat /.snapshots/3/info.xml [/B] 
<?xml version="1.0"?> 
<snapshot> 
  <type>pre</type> 
  <num>3</num> 
  <date>2021-03-17 03:02:59</date> 
  <description>apt</description> 
  <cleanup>number</cleanup> 
</snapshot> [/FONT]
[/FONT]

What's in snapshot?

:~# [B]vdir /.snapshots/3/snapshot/[/B] 
total 28 
drwxr-xr-x 1 root root    0 Jan  9  2020 backup 
lrwxrwxrwx 1 root root    7 Jan  5  2020 bin -> usr/bin 
drwxr-xr-x 1 root root  600 Mar 12 21:32 boot 
drwxr-xr-x 1 root root    0 Jan  5  2020 cdrom 
drwxr-xr-x 1 root root  140 Jan  3  2020 dev 
drwxr-xr-x 1 root root 5338 Mar 16 21:52 etc 
drwxr-xr-x 1 root root   10 Aug  6  2020 home 
....
drwxr-xr-x 1 root root    0 Jan  3  2020 srv 
drwxr-xr-x 1 root root    0 Dec  5  2019 sys 
drwxrwxrwt 1 root root 1950 Mar 16 22:02 tmp 
drwxr-xr-x 1 root root  126 Dec 31 13:08 usr 
drwxr-xr-x 1 root root  120 Feb 20 01:15 var

My entire system!
What is the size of it?

[FONT=monospace]:~# [B]btrfs fi usage /.snapshots/3/snapshot[/B] 
Overall: 
    Device size:                 465.76GiB 
    Device allocated:            134.03GiB 
    Device unallocated:          331.73GiB 
    Device missing:                  0.00B 
    Used:                        111.60GiB 
    Free (estimated):            352.90GiB      (min: 352.90GiB) 
    Data ratio:                       1.00 
    Metadata ratio:                   1.00 
    Global reserve:              206.77MiB      (used: 0.00B) 

Data,single: Size:131.00GiB, Used:109.83GiB (83.84%) 
   /dev/sda1     131.00GiB 
Metadata,single: Size:3.00GiB, Used:1.77GiB (59.12%) 
   /dev/sda1       3.00GiB 
System,single: Size:32.00MiB, Used:16.00KiB (0.05%) 
   /dev/sda1      32.00MiB 
Unallocated: 
   /dev/sda1     331.73GiB 
[/FONT]

Yup, it's the whole ball of wax! A complete snapshot.

Knowing this, one could use the "btrfs su snapshot" to create a rw copy of snapshot 3 and name it "@".
Where to put @? It's under the <ROOT_FS> but it cannot be accessed from within /. Remember? You cannot "cd .." from / and move to a higher directory level, which would be <ROOTT_FS>. That's why Snapper does NOT restore by renaming @ (or @home) to @old and creating a new @ from a snapshot. How does Snapper restore to a previous snapshot? By copying one or more (or all) files from a snapshot back into the system using it's "undochange" command. A "complete" restore is done by using just the "snapper undochange x..y" command. When finished the system is back at the condition it was in when it was "x". Changes will NOT include folders and files listed in filter.txt. To undo a specific file you'd use "snapper undochange x..y somefile1 somefile2 ...".

What won't Snapper restore?

[FONT=monospace][FONT=monospace][FONT=monospace][COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ [B]vdir /etc/snapper/ [/B][/COLOR]
total 0 
drwxr-xr-x 1 root root  8 Mar 17 11:12 configs 
drwxr-xr-x 1 root root 14 Mar 16 21:52 config-templates 
drwxr-xr-x 1 root root 44 Mar 16 21:52 filters 

[COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ [B]vdir /etc/snapper/filters/[/B] [/COLOR]
total 12 
-rw-r--r-- 1 root root 104 Feb  3  2020 base.txt 
-rw-r--r-- 1 root root  37 Feb  3  2020 lvm.txt 
-rw-r--r-- 1 root root  14 Feb  3  2020 x11.txt 

[COLOR=#000000]:[/COLOR][COLOR=#5454ff][B]~[/B][/COLOR][COLOR=#000000]$ [B]cat /etc/snapper/filters/base.txt [/B][/COLOR]
/etc/adjtime 
/etc/mtab 
/var/lib/logrotate.status 
/var/lib/misc/random-seed 
/var/lib/ntp/drift/ntp.drift
[/FONT][/FONT][/FONT]

And there, in base.txt, is what it won't restore. xll.txt holds "/.Xuathority".

So, say you do "sudo apt update && sudo apt full-upgrade". Snapper creates a "Pre" snapshot before apt runs "full-upgrade" and a "Post" snapshot after the upgrade is done. Say the upgrade borks your system in some way. IF the upgrade did't crash your display and you can still access the command line, you issue
"sudo snapper undochange x..y"
where x is the number of the Pre snapshot and y is the number of the Post snapshot. How do you get x and y?
By doing "sudo snapper list". Then you can use the undochange command.

There has been several times when I've inadvertantly erased a file or folder. Without Snapper's "undochange" comand how do I recover those files? By opening a Konsole, sudo'ing to root, and mounting my drive to /mnt. Then I fire mc and put the most recent snapshot in the left panel and my system in the right panel. I highlight the files/folders I want to restore and copy them from the left panel to the right panel. To roll back the entire system I mv @ to @old and create a rw @ from the most recent snapshot. Then I reboot.
Back in the system I delete @old.

Snapper does NOT backup your system. Neither does making snapshots the way I do. Your system is backed up ONLY when you use the btrfs send & receive commands to move a ro snapshot to a remote drive.

Snapper has a "diff" command. With all of its snapshots residing under / one can use Kdiff3 to view the difference between a folder in a snapshot and the same folder in another snapshot or in your system.
The btrfs send command also has that same ability, but only for entire snapshots.

btrfs send --no-data -p /mnt/snapshots/@202102182239 /mnt/snapshots/@202102192115 > diff.txt

Browse diff.txt to see the differences.

Unlike TimeShift, Snapper does NOT link your <ROOT_FS> to a subdirectory under /, which is dangerous.