Would like some assistance

[GreyGeek]

The script I wrote for snapshotting @ would have to be duplicated and in the second part @ is changed to @home, pointing to the their respective subdirectories under /mnt/snapshots/...

Here's the easy way.
https://www.kubuntuforums.net/forum/...-only-root-sub
My method was to sudo to root and mount /mnt. Then run mc and open @ /home in the left panel. In the right panel highlight jerry (@home/jerry) and use the copy command to copy jerry over to @/home. When it was done I edited /etc/fstab and put a # in front of the line that mounted @home to /home, and saved it. Then I unmounted everything and rebooted. Use your own account name, of course.

In that thread Oshunluver posted his method, which I think is better.

[Snowhog]

Thank you for the explanation as to why "I" get those errors.

The laptop is sacrificial; nothing critical on it. It's for playing/experimenting on.

I already figured out the "Don't do this part". When I plug in my external USB HDD, the Disk & Devices pop-up appears, but I (now) just ignore it. The drive is powered and identified by the system, as you point out. So that area of the script now reads:

# Before executing this script:
# Plug in external HDD. Ignore Disk & Devices pop-up.
# External drive will be powered and identified by system but not yet mounted.
#
echo "Mounting drives"
# mount internal HDD to /mnt
eval "mount /dev/disk/by-uuid/67fab1ac-b53c-4e10-9faf-f9a43941568e /mnt"
# mount external HDD to /backup
eval "mount /dev/disk/by-label/BACKUP /backup"

Given my setup; separate @ and @home; how would the following part of the script be written so it all works?

# capture the date and time
NOW=$(date +%Y%m%d%H%M)
echo "Making today's snapshot"
MKSNAP='btrfs su snapshot -r /mnt/@ /mnt/snapshots/@'$NOW
eval $MKSNAP
eval 'sync'
eval 'sync'

echo "Finding previous snapshot as parent "
PREVSNAP=""
list=($(ls /mnt/snapshots/))
PREVSNAP=${list[-2]}
NOW=${list[-1]}
echo "Attempting incremental backup"
if [[ -s "/mnt/snapshots/"$PREVSNAP ]];
then
MKINC='btrfs send -p /mnt/snapshots/'$PREVSNAP
MKINC=$MKINC' /mnt/snapshots/'$NOW
MKINC=$MKINC' | btrfs receive /backup'
echo $MKINC
eval $MKINC
eval 'sync'
eval 'sync'
eval "sync"
DELSNAP='btrfs subvol delete -C /mnt/snapshots/'${list[0]}
eval $DELSNAP
eval 'sync'
eval 'sync'
eval 'sync'
list=''
list=($(ls /backup))
DELSNAP='btrfs subvol delete -C /backup'${list[0]}
eval $DELSNAP
eval 'sync'
eval 'sync'
eval 'sync'
echo "Snapshots completed, oldest snapshots deleted"
eval 'umount /backup'
eval 'umount /mnt'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
echo "Drives unmounted and 3 syncs done"
else
echo 'Incremental backup failed using '$PREVSNAP' and '$NOW
echo 'Drives still mounted, ready to clean up'
fi

I know that's asking you to make the changes for me, and I hope you don't mind. It's that, or instruct me on how to merge (?) my @home into @ so my btrfs is setup like yours and the script would work as currently written.

[GreyGeek]

Don't do this part:

# Plug in external HDD. Select Mount in Disk & Devices pop-up
# External drive will be mounted to /media/paul/BACKUP
#

All you have to do is plug the external HDD in for it to be registered in /dev/disk/by-label as "BACKUP"

My script is for backing up just @, because I don't have a @home subvolume. I merged it into @/home and took out the line in fstab that mounted it. I wrote that script after I had merged the subvolumes.

The script you modified and ran tried to this:

btrfs send -p /mnt/snapshots/@202203060645 /mnt/snapshots/@home202203032257 | btrfs receive /backup

Notice that the parent is @202203060645 BUT the child is @home202203032257
That doesn't work because the list command:

list=($(ls /mnt/snapshots/))

doesn't not distinguish between @ and @home snapshots.

IF you are going to keep both @ and @home then I would create two separate subdirectories:
/mnt/snapshots/root
/mnt/snapshots/home
and store my @yyyymmddhhmm snapshots in the root subdir and @homeyyyymmddhhmm in the home subdir.
Then, add a section of code that picks out the root parent and root child, and ditto for the home parent and the home child.
That way the list command can properly identify the previous snapshot as parent.

[Snowhog]

GreyGeek I gave my external USB HDD a label (BACKUP) and made the necessary reference changes to the script.

I'm also unmounting the external USB HDD so I can then mount it at /backup

This is the script:

#!/bin/bash
# make_snapshot.sh

# Script to create backup snapshots to /mnt/snapshots and a differential backup to /backup
# To be run as root from /
#
# Before executing this script:
# Plug in external HDD. Select Mount in Disk & Devices pop-up
# External drive will be mounted to /media/paul/BACKUP
#
echo "Mounting drives"
# mount internal HDD to /mnt
eval "mount /dev/disk/by-uuid/67fab1ac-b53c-4e10-9faf-f9a43941568e /mnt"
# Unmount external HDD so it can be remounted to /backup
eval "umount /media/paul/BACKUP"
# mount external HDD to /backup
eval "mount /dev/disk/by-label/BACKUP /backup"

# capture the date and time
NOW=$(date +%Y%m%d%H%M)
echo "Making today's snapshot"
MKSNAP='btrfs su snapshot -r /mnt/@ /mnt/snapshots/@'$NOW
eval $MKSNAP
eval 'sync'
eval 'sync'

echo "Finding previous snapshot as parent "
PREVSNAP=""
list=($(ls /mnt/snapshots/))
PREVSNAP=${list[-2]}
NOW=${list[-1]}
echo "Attempting incremental backup"
if [[ -s "/mnt/snapshots/"$PREVSNAP ]];
then
MKINC='btrfs send -p /mnt/snapshots/'$PREVSNAP
MKINC=$MKINC' /mnt/snapshots/'$NOW
MKINC=$MKINC' | btrfs receive /backup'
echo $MKINC
eval $MKINC
eval 'sync'
eval 'sync'
eval "sync"
DELSNAP='btrfs subvol delete -C /mnt/snapshots/'${list[0]}
eval $DELSNAP
eval 'sync'
eval 'sync'
eval 'sync'
list=''
list=($(ls /backup))
DELSNAP='btrfs subvol delete -C /backup'${list[0]}
eval $DELSNAP
eval 'sync'
eval 'sync'
eval 'sync'
echo "Snapshots completed, oldest snapshots deleted"
eval 'umount /backup'
eval 'umount /mnt'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
echo "Drives unmounted and 3 syncs done"
else
echo 'Incremental backup failed using '$PREVSNAP' and '$NOW
echo 'Drives still mounted, ready to clean up'
fi

I start by plugging in my external USB HDD and selecting Mount in the Disk & Devices pop-up. Then I open a konsole and issue:

sudo -i
cd /
/make_snapshot.sh

This is the output from this morning:

paul@barley-cat:/$ sudo -i
root@barley-cat:~# cd /
root@barley-cat:/# /make_snapshot.sh
Mounting drives
Making today's snapshot
Create a readonly snapshot of '/mnt/@' in '/mnt/snapshots/@202203060645'
Finding previous snapshot as parent
Attempting incremental backup
btrfs send -p /mnt/snapshots/@202203060645 /mnt/snapshots/@home202203032257 | btrfs receive /backup
At subvol /mnt/snapshots/@home202203032257
At snapshot @home202203032257
ERROR: cannot find parent subvolume
Delete subvolume (commit): '/mnt/snapshots/@202203042120'
ERROR: Could not statfs: No such file or directory
Snapshots completed, oldest snapshots deleted
Drives unmounted and 3 syncs done

Why am I getting those two ERROR's?

[GreyGeek]

I always used the UUID and never thought about using the label. I have given the label "BACKUP" to my internal NVMe SSD and to an external USB HDD as well. They both cannot be mounted at the same time as "BACKUP". So, before I plug in my external USB SSD labeled "BACKUP" the listing is:

$ vdir /dev/disk/by-label/
total 0
lrwxrwxrwx 1 root root 15 Mar 4 07:26 BACKUP -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 15 Mar 4 07:26 DATA -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 10 Mar 4 07:26 EFI -> ../../sda1
lrwxrwxrwx 1 root root 10 Mar 4 07:26 ROOTFS -> ../../sda3
lrwxrwxrwx 1 root root 10 Mar 4 07:26 SWAP -> ../../sda2

AFTER I plug in the external SSD the listing shows:

$ vdir /dev/disk/by-label/
total 0
lrwxrwxrwx 1 root root 10 Mar 4 16:46 BACKUP -> ../../sdb1
lrwxrwxrwx 1 root root 15 Mar 4 16:46 DATA -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 10 Mar 4 16:46 EFI -> ../../sda1
lrwxrwxrwx 1 root root 10 Mar 4 16:46 ROOTFS -> ../../sda3
lrwxrwxrwx 1 root root 10 Mar 4 16:46 SWAP -> ../../sda2
~$

The NVMe is hidden. If I were to use

$ sudo -i
# mount BACKUP /backup                      
mount: /backup: special device BACKUP does not exist.
# mount /dev/disk/by-label/BACKUP /backup
# vdir /backup
total 0
drwxr-xr-x 1 root root 334 Feb 18 20:55 @202202182046
drwxr-xr-x 1 root root 334 Feb 23 19:11 @202202231903
drwxr-xr-x 1 root root 334 Feb 24 20:57 @202202242057
drwxr-xr-x 1 root root 334 Feb 25 19:18 @202202251913
drwxr-xr-x 1 root root 334 Feb 26 20:41 @202202262040
drwxr-xr-x 1 root root 334 Feb 28 16:10 @202202281609
drwxr-xr-x 1 root root 334 Mar  1 18:15 @202203011815
drwxr-xr-x 1 root root 334 Mar  2 22:13 @202203022213

The external drive gets mounted.

/dev/sdb1 on /backup type btrfs (rw,relatime,space_cache,subvolid=5,subvol=/)

So, I could mount the internal BACKUP or the external backup using

mount /dev/disk/by-label/BACKUP /backup

and which one gets mounted would be determined by plugging in the external SSD or not.
Nice!


PS: The UUID mistype plagued me until I restored to just using blkid in the Konsole and then C&P to put it onto the command line or into a script. Using the BACKUP label is going to change all of that. I can run the same script, first without the external SSD plugged in, and then with the external SSD plugged in.

[Snowhog]

GreyGeek Okay, after changing the HDD IDs to reflect mine, and commenting out the mounting line for the external HDD (as already explained, I accept mounting when it is plugged in), I ran the script. The results:

paul@barley-cat:~$ sudo -i
root@barley-cat:~# cd /
root@barley-cat:/# /make_snapshot.sh
Mounting drives
mount: /mnt: special device /dev/disk/by-uuid/67fab1ac-b53c-4e10-9faf-f9a434156e does not exist.
Making today's snapshot
Create a readonly snapshot of '/mnt/@' in '/mnt/snapshots/@202203041643'
Finding previous snapshot as parent
Attempting incremental backup
btrfs send -p /mnt/snapshots/@202203041643 /mnt/snapshots/@home202203032257 | btrfs receive /backup
At subvol /mnt/snapshots/@home202203032257
At snapshot @home202203032257
ERROR: parent subvol is not reachable from inside the root subvol
Delete subvolume (commit): '/mnt/snapshots/@202203032257'
ERROR: Not a Btrfs subvolume: Invalid argument
Snapshots completed, oldest snapshots deleted
umount: /backup: not mounted.
Drives unmounted and 3 syncs done
root@barley-cat:/#

The unmount: /backup: not mounted is my bad. I didn't change the eval 'umount /backup' line.

As to the to ERROR lines. Given my setup, what do I make of them?
The mount: /mnt: error is my bad. Didn't write out the UUID correctly.

[Snowhog]

jlittle The drive doesn't have a label, so the UUID is identified.

[jlittle]

If I accept the Disk & Devices option to Mount and Open the external HDD, it will always be mounted at /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969.

If the volume has a label, I would expect it to be mounted in /media/paul with that label, rather than the UUID.

[GreyGeek]

It is save at:

$ sudo vdir /
[sudo] password for jerry:
total 40
drwxr-xr-x 1 root root 0 Jan 3 16:23 backup
lrwxrwxrwx 1 root root 7 Oct 28 08:30 bin -> usr/bin
drwxr-xr-x 1 root root 1422 Feb 24 21:01 boot
drwxr-xr-x 22 root root 4580 Mar 4 07:26 dev
drwxr-xr-x 1 root root 4562 Mar 2 15:29 etc
-rw-r--r-- 1 root root 515 Jan 5 2020 he-ipv6.service
drwxr-xr-x 1 root root 10 Jan 3 16:28 home
lrwxrwxrwx 1 root root 7 Oct 28 08:30 lib -> usr/lib
lrwxrwxrwx 1 root root 9 Oct 28 08:30 lib32 -> usr/lib32
lrwxrwxrwx 1 root root 9 Oct 28 08:30 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 10 Oct 28 08:30 libx32 -> usr/libx32
-rwxr-xr-x 1 root root 1600 Feb 8 15:58 make_snapshot.sh
drwxr-xr-x 1 root root 10 Jan 2 14:59 media
drwxr-xr-x 1 root root 0 Oct 28 08:30 mnt
-rwxr-xr-x 1 root root 343 Feb 8 15:59 mount_drives.sh
...

I have two internal SSD's and a host of external HDDs & and an SSD.
I plug my external 1Tb spinner (which was the HDD that came with this new HP and I put it into a USB HD Caddy) and then run the script using:

sudo -i
cd /
/make_snapshots.sh

The script umounts my external USB HDD and /mnt when it is done.

Using the history command, I the use !nnnn to run the most recent command to mount my NVMe 1Tb SSD to /backup and the system to /mnt. Then I issue

vdir /mnt/snapshots

and

vdir /backup

to see what's on them. I pick the most recent snapshot that is the same on both to use as the parent, and then choose the newest snapshot on /mnt/snapshots as the child, the snapshot I want to send to /backup.

send -p /mnt/snapshots/@parentyyyymmddhhss /mnt/snapshots/@mostrecentsnapshotyyyymmddhhss | btrfs receive /backup

Then I use

btrfs filesystem sync /backup

two or three times, or until it returns immediately, indicating nothing was done. (You can also use 'sync')

Then I umount /mnt and /backup and then exit root and then exit the Konsole.
I use the NVMe SSD less frequently than I do the USB Caddy containing the 1Tb spinner, sometimes only once a week or so. I've used the send command sans -p a couple times and it takes about 25 minutes to send my snapshot to the NVMe SSD. I do that if I don't have a "parent" snapshot on the NVMe.

So, all in all, with the aid of the history command and the ability to execute previous history command using "!nnnn" I've pretty much been doing my snapshots manually. In all I have four USB drives that I send backup snapshots to, and some 128Gb and 256Gb USB sticks as well. For a long time I always kept a LiveUSB stick of Kubuntu in my pants watch pocket and a USB stick containing a recent snapshot of my system. Since I've combined @home into @, I only need to track and keep one snapshot and have no risk of mixing up the @ with the wrong @home.

[Snowhog]

Here is the script that I use when I don't do it manually.

You run this script from / as root, but where do you have the script saved at?

So, your script, it creates snapshots that are kept on your internal HDD (/mnt/snapshots) and incremental snapshots that are kept on the external HDD (/backup)?

If that is correct, how are you actually protected if your internal HDD fails?

[GreyGeek]

So I had a thought, and no, it didn't hurt much.

If I accept the Disk & Devices option to Mount and Open the external HDD, it will always be mounted at /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969. I can take advantage of that and use:

btrfs send /mnt/snapshots/@yyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969
btrfs send /mnt/snapshots/@homeyyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969

Mmm.. never used that approach. Interesting.

Those times aren't bad, but once you have a "parent" on your backup drive the incremental send is extremely fast. In your case I'd guess that it would complete in under 5 seconds.

[Snowhog]

So I had a thought, and no, it didn't hurt much.

If I accept the Disk & Devices option to Mount and Open the external HDD, it will always be mounted at /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969. I can take advantage of that and use:

btrfs send /mnt/snapshots/@yyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969
btrfs send /mnt/snapshots/@homeyyyymmddhhmm | btrfs receive /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969

[GreyGeek]

Using the blkid requires the use of "/dev/disk/by-uuid/..." as in

mount /dev/disk/by-uuid/038bd596-59b4-4bb0-98e5-3cbeec1dd339 /mnt

The blkid command gives a list of UUID's for attached devices. There are several paths that can be used.

$ vdir /dev/disk
total 0
drwxr-xr-x 2 root root 320 Mar  4 07:26 by-id
drwxr-xr-x 2 root root 140 Mar  4 07:26 by-label
drwxr-xr-x 2 root root 140 Mar  4 07:26 by-partuuid
drwxr-xr-x 2 root root 180 Mar  4 07:26 by-path
drwxr-xr-x 2 root root 140 Mar  4 07:26 by-uuid

The /dev/sdXn path of a device can change randomly without warning. That's why UUID is used. The same UUID points to the /dev/sdXn path as it currently is following bootup. When mounting manually using /dev/sdXn is fine because it is determined at bootup or plugin. Inside a script using /dev/sdXn can fail because /dev/sdXn may not point to the same device. When first testing btrfs I used a raid1 using two devices, sda1 and sda2. When I plugged another SATA drive into my Acer and booted up I expected it to be sdc1. To my shock I discovered that my raid1 was sda1 and sdc1, even though they were the same two devices. The new SATA drive was sdb1.

Here are the paths given in MY /dev/disk/by-uuid directory

$ vdir /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 15 Mar  4 07:26 2b08ee29-bf29-4b5c-bb40-6aafaf66cec8 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 15 Mar  4 07:26 4ea5991d-7da6-4387-907d-acef20d047de -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 10 Mar  4 07:26 83cb8dfd-aadc-4558-884d-eeafac2303bf -> ../../sda2
lrwxrwxrwx 1 root root 10 Mar  4 07:26 9bc19383-57db-4a32-938d-8466a133d94d -> ../../sda3
lrwxrwxrwx 1 root root 10 Mar  4 07:26 C853-95B5 -> ../../sda1

The paths following the "-->" are determined at boot, but the UUID is always the same for the same device. That's why I use the UUID in my scripts but I use the /dev/sdXn manually.

[Snowhog]

Correct. Typo on my part. Fixed it in the post so that others won't trip over it.

How did your first snapshots go? How long did they take to send to the external HDD?

Thanks for confirming and correcting the typo.

I haven't done a snapshot yet. I'd like to understand why my external HDD is being automounted. I get that that is likely the default configuration for this action in KDE Plasma. But it does make following your steps 'complicated'. When plugged in, it is being mounted to /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969. I guess I can just issue:

umount /media/paul/7baf740a-eab6-40ce-9225-a99a1d0ee969
mount /dev/sdb1 /backup

I just tried that, and that does work.

I've got /dev/sda1 mounted to /mnt
I've got /dev/sdb1 mounted to /backup
I created /mnt/snapshots
I created snapshots of @ and @home in /mnt/snapshots
I'm in the process of sending the @ snapshot to the external drive. When it's finished, I'll send @home.
Both sent/received without errors. Sweet.

The process of sending @ didn't take very long; five+ minutes I think. @home took even less time. This isn't a production laptop, so there isn't much on it other than the Kubuntu 22.04 install.

[GreyGeek]

That's much more helpful (to me). Thank you.

First Issue:

mount /dev/sdb /backup

/backup didn't exist, so I created it on /dev/sda1

Second Issue:

When I plug in my external HDD, the Disk & Devices pop-up notifier appears. I just close the pop-up; I don't click on Mount and Open. In the konsole I type:

mount /dev/sdb /backup

and am told:

mount: /backup: wrong fs type, bad option, bad superblock on /dev/sdb, missing codepage or helper program, or other error.

If I then click on Mount and Open in Disk & Devices, the drive gets mounted and Dolphin opens showing the drive in a separate tab (465.8 GiB Removable Media). If I then repeat the mount command in konsole I get:

mount: /backup: /dev/sdb already mounted or mount point busy.



Update:

After the external HDD was mounted via Disk & Devices, I opened KDE Partition Manager. I selected /dev/sdb and highlighted /dev/sdb1 and right-clicked and selected Unmount.

Opened Konsole and typed:

sudo -i
mount /dev/sdb1 /backup

That worked.

So your instruction: mount /dev/sdb /backup is wrong? Needs to be /dev/sdb1?

Correct. Typo on my part. Fixed it in the post so that others won't trip over it.

How did your first snapshots go? How long did they take to send to the external HDD?