Would like some assistance

[GreyGeek]

Nice code, jlittle !! I'm going to steal it and add it to my script!
Snowhog , how long did it take to send to full snapshots to your remote HDD

[Snowhog]

jlittle Thank you! Getting it 'brief' wasn't the intended goal, but seeing how it can be done 'educates' me, and that is always a plus. I knew there were folks here who could improve on what I had attempted!

[jlittle]

The DURATION variable almost certainly can be written better, but I spent a reasonable amount of time trying to do so, and failed. Using the TIMESTART, TIMESTOP, STARTCLOCK, and STOPCLOCK variables was the only way I could get it to work. Maybe smarter folk here can see how to do it 'cleaner'.

Shell arithmetic is better for a difference. (Using -u may cause grief if you happen to be running at UTC midnight, which would be 6 pm in Andover, MN; one must use -u with -d @.)

STARTCLOCK=$(date +%s)
# run stuff...
STOPCLOCK=$(date +%s)
SECS=$((STOPCLOCK - STARTCLOCK))
DURATION=$(date -d "0 $SECS sec" +%H:%M:%S)

The last three lines could be merged to one, if you prefer brevity to readability:
DURATION=$(date --date "0 $(($(date +%s) - STARTCLOCK)) sec" +%H:%M:%S)

Nifty to use date --d with a date of "0 somenumber sec" and then a format with only the time. The "0" means today at 00:00. If that's too clever, or the job might run over 24 hours,
DURATION=$(printf "%02d:%02d:%02d" $((SECS / 3600)) $((SECS % 3600 / 60)) $((SECS % 60)) )

[Snowhog]

Made some cosmetic changes. The script works. As already stated, it just makes full snapshots of my @ and @home subvolumes and then sends them to my external USB HDD. But that's enough for now. Here is the 'latest version' of the script:

#!/bin/bash
# snapshot.sh
#
# Variable declarations
#
NOW=$(date +%Y%m%d%H%M)
MKSNAP1="btrfs su snapshot -r /mnt/@ /mnt/snapshots/@$NOW"
MKSNAP2="btrfs su snapshot -r /mnt/@home /mnt/snapshots/@home$NOW"
SNDSNAP1="btrfs send /mnt/snapshots/@$NOW | btrfs receive /backup"
SNDSNAP2="btrfs send /mnt/snapshots/@home$NOW | btrfs receive /backup"
TIMESTART=$( date '+%H:%M:%S' )
STARTCLOCK=$(date -u -d "$TIMESTART" +"%s")

# To be run as root from /
#
# Before executing this script:
# Plug in external HDD. Ignore Disk & Devices pop-up.
# External drive will be powered and identified by system but not yet mounted.
#
echo "Mounting drives"
# mount internal HDD to /mnt
eval "mount /dev/disk/by-label/LAPTOP /mnt"
echo "Internal HDD mounted to /mnt"
# mount external HDD to /backup
eval "mount /dev/disk/by-label/USBHDD /backup"
echo "External USB HDD mounted to /backup"
echo ""
# capture current date and time
echo "Timestamp: $NOW"
echo ""
# Create snapshots
echo "Making today's snapshot of @"
eval $MKSNAP1
eval 'sync'
eval 'sync'
echo "@$NOW successfully created"
echo ""
echo "Making today's snapshot of @home"
eval $MKSNAP2
eval 'sync'
eval 'sync'
echo "@home$NOW successfully created"
echo ""
# Send snapshots
echo "Sending @$NOW and @home$NOW to external USB HDD"
eval $SNDSNAP1
eval 'sync'
eval 'sync'
eval $SNDSNAP2
eval 'sync'
eval 'sync'
TIMESTOP=$( date '+%H:%M:%S' )
STOPCLOCK=$(date -u -d "$TIMESTOP" +"%s")
DURATION=$(date -u -d "0 $STOPCLOCK sec - $STARTCLOCK sec" +"%H:%M:%S" )
echo "@$NOW and @home$NOW successfully sent to external USB HDD"
echo ""
echo "Creating and Sending Snapshots took: $DURATION"
echo ""
# Show snapshots on Internal and External USB HDD
echo "Contents of /mnt/snapshots"
vdir /mnt/snapshots
echo ""
echo "Contents of /backup"
vdir /backup
# echo ""
echo "Unmounting drives"
eval 'umount /backup'
eval 'umount /mnt'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
echo ""
echo "Drives unmounted and 3 syncs done"

The DURATION variable almost certainly can be written better, but I spent a reasonable amount of time trying to do so, and failed. Using the TIMESTART, TIMESTOP, STARTCLOCK, and STOPCLOCK variables was the only way I could get it to work. Maybe smarter folk here can see how to do it 'cleaner'.

[GreyGeek]

Because, typically, an incremental snapshot usually takes less than 20 seconds, while full snapshots can take 15-20 minutes. The result is the same. My internal backup is a 1Tb NVMe SSD. Its NVM Cntl chip has a maximum T of 70C and it begin throttling at 68.8C. The NVMe VNAND chip starts heating up slowly but continues heating up after the snapshot is done and can get as high as 68.8 as well. I use incremental snapshots to minimize the heating and because they take significantly less time. I have a heat sink on the NVMe but because of the restricted space it is only 2mm thick. I tried a 4mm thick heat sink and it kept the T at or below 52C, but it was so thick that I couldn't completely close the back cover.

BTW, comments do not show up on the "New Posts" or "Today's Posts" listings.

[Snowhog]

GreyGeek Why do you only send incremental snapshots to /backup?

[GreyGeek]

Here is where I got my list foo from. It involves the use of curly brackets and square brackets on bash variables filled with the ls command.
https://www.linux.com/topic/desktop/...y-braces-bash/

[Snowhog]

Unlike GreyGeeks use case, I don't have a need for such a detailed backup routine. The laptop isn't my production PC. What I'm really interested in is what I've accomplished so far, and that's creating full snapshots of @ and @home, and then sending them to an external storage device; my external USB HDD.

So, my current question is this: As I have successfully created and sent full snapshots to my external USB HDD, and I'm not (at this time anyway) going to be creating incremental snapshots, is there any reason (other than convenience and speed) for keeping the snapshots that are on my internal HDD?

[Snowhog]

Yes, I'm aware of the problem my schema introduces in regards keeping both @ and @home, and not having separate directories for @ and @home snapshots in both, /mnt/snapshots and /backup. But for now, this is just for learning. Baby Steps.

I haven't included the process for creating incremental snapshots yet either. Again, Baby Steps.

[Snowhog]

jlittle I'm just using what GreyGeek wrote as my starting point. As I said in my first post, btrfs is voodoo magic to me. I learn better when I have a working example. His script works for him, and my current rendition of it works for me. Can it be improved on? Most certainly! Can it be fine tuned to use the least amount of resources? Again, most certainly.

I'm all for having a 'lean, mean machine', so if you have suggestions on how I can improve my endeavor, please don't hesitate to suggest. (Examples most welcome!)

[GreyGeek]

That looks OK to me as for a coding is concerned.

However, your code still puts both @yyyymmddhhmm and @homeyyyyddhhmm into the same directory. Five days of snapshots could look something like this:

.../snapshots/@yyyymmd-5hhmm
.../snapshots/@yyyymmd-4hhmm
.../snapshots/@yyyymmd-3hhmm
.../snapshots/@yyyymmd-2hhmm
.../snapshots/@yyyymmd-1hhmm
.../snapshots/@yyyymmdhhmm
.../snapshots/@homeyyyymmd-5hhmm
.../snapshots/@homeyyyymmd-4hhmm
.../snapshots/@homeyyyymmd-3hhmm
.../snapshots/@homeyyyymmd-2hhmm
.../snapshots/@homeyyyymmd-1hhmm
.../snapshots/@homeyyyymmdhhmm

OR, hopefully, they could like something like this

.../snapshots/@yyyymmd-5hhmm
.../snapshots/@homeyyyymmd-5hhmm
.../snapshots/@yyyymmd-4hhmm
.../snapshots/@homeyyyymmd-4hhmm
.../snapshots/@yyyymmd-3hhmm
.../snapshots/@homeyyyymmd-3hhmm
.../snapshots/@yyyymmd-2hhmm
.../snapshots/@homeyyyymmd-2hhmm
.../snapshots/@yyyymmd-1hhmm
.../snapshots/@homeyyyymmd-1hhmm
.../snapshots/@yyyymmdhhmm
.../snapshots/@homeyyyymmdhhmm

The list commands in my code which identifies the oldest snapshot in a folder are

list=''
list=($(ls /mnt/snapshots ))
DELSNAP='btrfs subvol delete -C /mnt/snapshots/'${list[0]}
....
list=''
list=($(ls /backup))
DELSNAP='btrfs subvol delete -C /backup/'${list[0]}

Using the second listing the deletion code could be:

list=''
list=($(ls /mnt/snapshots ))
DELSNAP='btrfs subvol delete -C /mnt/snapshots/'${list[0]}
list=''
list=($(ls /mnt/snapshots ))
DELSNAP='btrfs subvol delete -C /mnt/snapshots/'${list[0]}
list=''
list=($(ls /mnt/snapshots ))
DELSNAP='btrfs subvol delete -C /backup/'${list[0]}
list=''
list=($(ls /mnt/snapshots ))
DELSNAP='btrfs subvol delete -C /backup/'${list[0]}

My list foo isn't good enough to remove the two oldest listings in the first set. Now you know why I recommend saving @ snapshots in .../snapshots/root/
and @home snapshots in
.../snapshots/home/

[jlittle]

Why all the "evals"? Using them stops your editor from properly syntax colouring the script. evals can be treacherous.

I wonder why

MKSNAP1="btrfs su snapshot -r /mnt/@ /mnt/snapshots/@$NOW"
eval $MKSNAP1

and not just

btrfs su snapshot -r /mnt/@ /mnt/snapshots/@$NOW

All those syncs look excessive. Some of my external drives are active for 20 or 30 seconds after my backup script finishes, while several GB in cache are flushed to the drives, but IIRC unmounts stall till the flush is complete. I wonder if a btrfs filesystem sync /backup is a good idea.

[Snowhog]

GreyGeek I've been working on this (script) since last night, and have what I believe to be correct (for what I want "at this time"). I'd like your critical eye on it to see if you notice anything wrong.

This script doesn't deal with incremental snapshots; maybe later; and it doesn't (yet) deal with cleaning up (deleting) older snapshots. That will be next. Baby Steps. First things first.

#!/bin/bash
# snapshot.sh
#
# To be run as root from /
# Type: /snapshot.sh
# To get the time taken to complete, use: time
# Type: time /snapshot.sh
#
# Before executing this script:
# Plug in external HDD. Ignore Disk & Devices pop-up. # External drive will be powered and identified by system but not yet mounted.
#
echo "Mounting drives"
# mount internal HDD to /mnt
eval "mount /dev/disk/by-label/LAPTOP /mnt"
echo "Internal HDD mounted to /mnt"
# mount external HDD to /backup
eval "mount /dev/disk/by-label/BACKUP /backup"
echo "External USB HDD mounted to /backup"
echo ""
# capture current date and time
NOW=$(date +%Y%m%d%H%M)
echo "Timestamp is $NOW"
echo ""
# Create snapshots
echo "Making today's snapshot of @"
MKSNAP1="btrfs su snapshot -r /mnt/@ /mnt/snapshots/@$NOW"
eval $MKSNAP1
eval 'sync'
eval 'sync'
echo "@$NOW successfully created"
echo ""
echo "Making today's snapshot of @home"
MKSNAP2="btrfs su snapshot -r /mnt/@home /mnt/snapshots/@home$NOW"
eval $MKSNAP2
eval 'sync'
eval 'sync'
echo "@home$NOW successfully created"
echo ""
# Send snapshots
SNDSNAP1="btrfs send /mnt/snapshots/@$NOW | btrfs receive /backup"
SNDSNAP2="btrfs send /mnt/snapshots/@home$NOW | btrfs receive /backup"
echo "Sending @$NOW and @home$NOW to external USB HDD"
eval $SNDSNAP1
eval 'sync'
eval 'sync'
eval $SNDSNAP2
eval 'sync'
eval 'sync'
echo "@$NOW and @home$NOW successfully sent to external USB HDD"
echo ""
# Show snapshots on External USB HDD
vdir /backup
echo ""
echo "Unmounting drives"
eval 'umount /backup'
eval 'umount /mnt'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
eval 'sync'
eval 'sleep 1'
echo ""
echo "Drives unmounted and 3 syncs done"

Added:
I ran this and it completed without any errors. Sweet!

[GreyGeek]

I've tried TimeShift and dropped it for reasons discussed in this thread.
https://www.kubuntuforums.net/forum/...-timeshift-yet

[tomcloyd]

Arrrgh...this thread is convincing me to stick with Timeshift for backups. When the new Kubuntu LTS version is out I well may install with btrfs - haven't decided. Timeshift can do btrfs backups.