Would like some assistance

[oshunluvr]

My script:

1. Runs every day at 5:30 AM (or when I turn on the PC) via anacron.
2. Makes a daily snapshot of the install and home subvolumes.
3. Makes the snapshots as RW so they can be easily used but sets the backup snapshots as RO so they can be "sent."
4. Resets the time/date stamp of the snapshot so it's obvious when the snapshot was taken.
5. Snapshots are rotated on a seven day cycle, meaning when a snapshot is taken on Tuesday, last Tuesday's snapshot is deleted.
6. Every Sunday:
   1. The last snapshot is sent incrementally to a backup on a different drive.
   2. The previous "parent" (for incremental backups) backup is deleted and the new one "elevated" to parent status.
7. Sends a desktop notification to me that the daily task was completed.
8. Does all the above on the root and home subvolumes.

This results in snapshots going back 7 days of PC usage and a backup that is updated weekly. For example, I was gone from Jan 25th to March 4th. Here's my current snapshots folder:

stuart@office:~$ ll /subvol/snapshots/
total 16K
drwxr-xr-x 1 root root 736 Mar  5 05:40  ./
drwxr-xr-x 1 root root 256 Mar  4 17:35  ../
drwxr-xr-x 1 root root 178 Jan 13 05:39 '@KDEneon_backup'/
drwxr-xr-x 1 root root 178 Jan 21 05:37 '@KDEneon_backup-new'/
drwxr-xr-x 1 root root 178 Mar  5 05:40 '@KDEneon_daily_0'/
drwxr-xr-x 1 root root 178 Mar  4 14:28 '@KDEneon_daily_1'/
drwxr-xr-x 1 root root 178 Jan 26 05:37 '@KDEneon_daily_2'/
drwxr-xr-x 1 root root 178 Jan 25 05:38 '@KDEneon_daily_3'/
drwxr-xr-x 1 root root 178 Jan 24 08:02 '@KDEneon_daily_4'/
drwxr-xr-x 1 root root 178 Jan 23 05:37 '@KDEneon_daily_5'/
drwxr-xr-x 1 root root 178 Jan 22 05:36 '@KDEneon_daily_6'/
drwxr-xr-x 1 root root  32 Jan 13 05:39 '@KDEneon_home_backup'/
drwxr-xr-x 1 root root  32 Jan 21 05:37 '@KDEneon_home_backup-new'/
drwxr-xr-x 1 root root  32 Mar  5 05:40 '@KDEneon_home_daily_0'/
drwxr-xr-x 1 root root  32 Mar  4 14:28 '@KDEneon_home_daily_1'/
drwxr-xr-x 1 root root  32 Jan 26 05:37 '@KDEneon_home_daily_2'/
drwxr-xr-x 1 root root  32 Jan 25 05:38 '@KDEneon_home_daily_3'/
drwxr-xr-x 1 root root  32 Jan 24 08:03 '@KDEneon_home_daily_4'/
drwxr-xr-x 1 root root  32 Jan 23 05:37 '@KDEneon_home_daily_5'/
drwxr-xr-x 1 root root  32 Jan 22 05:36 '@KDEneon_home_daily_6'/
​

You can see by the filedates:
"...daily_0" is today's snapshot.
"...backup-new" is the last backup sent incrementally.
"...backup" is the previous backup kept to allow incremental sending of the new backup.

Notes:

• The "Mar 4" snapshot was taken at 2:28 PM - that's when I returned to my office and powered up my PC.
• The reason for number 4 above is a snapshot will use the time/date of the original subvolume. This results in all snapshots having identical time/date stamps. I change them to the current date/time so I can more easily tell when the actions occurred.
• The default time for anacron to begin is 7:30 AM. If you want something different do "sudo systemctl edit --full anacron.timer" and make the edit. I changed mine to "OnCalendar=*-*-* 05..23:30" so it's starts at 5:30 am.

In theory, you could just change the subvolume names and mounts points in the script to match your system and place it in /etc/cron/daily and it would start up right away. I did my best to make it "transportable" so I could use if on other PC's I have and work on.

Script contents:

#!/bin/bash
#
# IMPORTANT:
# This script requires the root btrfs file system be mounted somewhere before running.
# Alternately, have the mount declared in fstab and the script will mount it.
#
exec 1> >(logger -s -t $(basename $0)) 2>&1 # Log the script activity

## Set the variables. Edit this section as needed
# declare an array variable of all target subvolumes
declare -a SUBVLIST=("@KDEneon" "@KDEneon_home")

# other variables
NOTIFYUSER="stuart"                     # the username to get error messages
NOTIFYDISPLAY=":0.0"                    # the above user's X session
source_folder="/subvol/"                # Root file system mount location
backup_folder="/mnt/root_backup/"       # Backup file system mount location
snapshot_folder="/subvol/snapshots/"    # Snapshots folder
addname="_daily_"                       # This is added to the subvolume name of the daily snapshots
lastsnapnum=6                           # Last snapshot number to be saved before moving to backup status

# Verify the root BTRFS file system is mounted and mount it if not, or fail;
if [ -d "$source_folder" ]; then
    if ! mountpoint -q "$source_folder"; then
        mount "$source_folder"
        if ! mountpoint -q "$source_folder"; then
            su $NOTIFYUSER -c "export DISPLAY=${NOTIFYDISPLAY}; /usr/bin/notify-send -i ksnapshot -t 0 'Daily snapshots:' 'Daily snapshot operation failed - no mountable subvolume folder.'"
            exit 1
        fi
    fi
    else
        su $NOTIFYUSER -c "export DISPLAY=${NOTIFYDISPLAY}; /usr/bin/notify-send -i ksnapshot -t 0 'Daily snapshots:' 'Snapshot operation failed - subvolume folder incorrect.'"
        exit 1
fi

## Begin snapshot process
# loop through the list of subvolumes
for SUBV in "${SUBVLIST[@]}"; do
    SUBVPATH="$source_folder""$SUBV"
    SUBVSNAP="$snapshot_folder""$SUBV"

# Move last daily snapshot to backup status
    if [[ -d "$SUBVSNAP""$addname""$lastsnapnum" ]]; then
        btrfs su de -c "$SUBVSNAP""_backup-new"
        mv "$SUBVSNAP""$addname""$lastsnapnum" "$SUBVSNAP""_backup-new"
        btrfs pr set -ts "$SUBVSNAP""_backup-new" ro true
    fi

# Roll the current snapshot names by adding 1 to each trailing number
    for ((i="$lastsnapnum-1";i>=0;i--)); do
        if [[ -d "$SUBVSNAP""$addname"$i ]]; then
            mv "$SUBVSNAP""$addname"$i "$SUBVSNAP""$addname"$(($i+1));
        fi
    done

# Take new read-right snapshot
    btrfs su sn "$SUBVPATH" "$SUBVSNAP""$addname""0"
    touch "$SUBVSNAP""$addname""0"
done

# Notify the user that the job is done
su $NOTIFYUSER -c "export DISPLAY=${NOTIFYDISPLAY}; /usr/bin/notify-send -i ksnapshot -t 0 'Daily snapshots:' 'Snapshot operation complete.'"

#exit 0

# If it's Sunday, do a backup
if [[ $(date +%u) -ne 7 ]] ; then
echo $(date +%A)
# Script complete
    exit 0
fi

# Verify the backup file system is mounted and mount it if not, or fail;
if [ -d "$backup_folder" ]; then
    if ! mountpoint -q "$backup_folder"; then
        mount "$backup_folder"
        if ! mountpoint -q "$backup_folder"; then
            su $NOTIFYUSER -c "export DISPLAY=${NOTIFYDISPLAY}; /usr/bin/notify-send -i ksnapshot -t 0 'Weekly backup:' 'Backup operation failed - no mountable subvolume folder.'"
            exit 1
        fi
    fi
    else
        su $NOTIFYUSER -c "export DISPLAY=${NOTIFYDISPLAY}; /usr/bin/notify-send -i ksnapshot -t 0 'Weekly backup:' 'Backup operation failed - subvolume folder incorrect.'"
        exit 1
fi

## Begin backup process
echo "Starting backup..."
# loop through the list of subvolumes
for SUBV in "${SUBVLIST[@]}"; do
     SUBVSNAPRO="$snapshot_folder""$SUBV""_backup"
     SUBVBACK="$backup_folder""$SUBV""_backup"
    
# Send incremental backup
    btrfs send -p "$SUBVSNAPRO" "$SUBVSNAPRO""-new" | btrfs receive "$backup_folder"
# Remove old snapshot and backup
    btrfs su de -c "$SUBVSNAPRO"
    btrfs su de -c "$SUBVBACK"
# Move new backups to old backup status
#    if [[ -d "$SUBVBACK" ]]; then
    mv "$SUBVBACK""-new" "$SUBVBACK"
#    fi
#    if [[ -d "$SUBVSNAPRO" ]]; then
    mv "$SUBVSNAPRO""-new" "$SUBVSNAPRO"
#    fi
done

# Notify the user that the job is done
su $NOTIFYUSER -c "export DISPLAY=${NOTIFYDISPLAY}; /usr/bin/notify-send -i ksnapshot -t 0 'Weekly backup:' 'Backup operation complete.'"

# Script complete
exit 0
​

[Snowhog]

Sorry I didn't see them. I still strongly suspect that the problem with the SNAPLIST= lines is in the script before them.

As I said in post #75, I have the script working. I was trying to use sh instead of bash. Using sh, the error occurs. Using bash, the script runs flawlessly.

[jlittle]

The quotes precede the variable; it puts a space before the echoed output.

Sorry I didn't see them. I still strongly suspect that the problem with the SNAPLIST= lines is in the script before them.

[Snowhog]

jlittle If you are referring to these type lines:

echo " "$DELSNAP1
echo " "$DELSNAP2

The quotes precede the variable; it puts a space before the echoed output.

[Snowhog]

jlittle Yes, the script I'm trying to use is the same as what is posted in post #63. I got the script to run to completion just a bit ago, with no errors. As root (sudo -i), I ran: bash snapshot.sh No errors. Took a bit over 18-minutes to create and send to the backup drive, my @ and @home directories.

When I was receiving the error on the variable definition line, I was launching the script with sh ./snapshot.sh So a difference in how sh and bash handle things. The script file is set executable, so I should be able to run it directly: snapshot.sh I'll try that next time.

You have to keep your eyes open and be aware of nuances! They're pesky little buggers!

[jlittle]

Is the script in post #63 still the current version? I expect not, because the "DELSNAP" lines have unclosed quotes.

[Snowhog]

There's nothing wrong with SNAPLIST1=($(ls /mnt/snapshots/|grep 'ROOT')) as far as I can see.

First, I created /mnt/snapshots and reran the script this morning. Bash still complains about the first of these two lines with the same ./snapshot.sh: 89: Syntax error: "(" unexpected

I tested this line manually and it works; bash doesn't complain. I'm at a loss at this point (but still searching the 'Net) as to why it isn't liked when executed within the script.

[Snowhog]

The error may have been due to my not having created the snapshot directory in /mnt. /mnt/snapshot has to exist before running the script. It’s there now. I’ll test later.

[jlittle]

I recall an error like that being due to a problem many lines previously. There's nothing wrong with SNAPLIST1=($(ls /mnt/snapshots/|grep 'ROOT')) as far as I can see.

[Snowhog]

So there were changes made in bash version 5.2. At the time I wrote/used my script, bash version was 5.1.

New Features
============

This is an update to the fifth major release of bash.

Read the file NEWS in the bash-5.2 distribution for a complete description
of the new features. A copy of the relevant portions is included below.

This release fixes several outstanding bugs in bash-5.1 and introduces
a number of new features.

There are a number of bug fixes, including several bugs that caused the
shell to crash. Complete details are available in the CHANGES file.

The most notable new feature is the rewritten command substitution parsing
code, which calls the bison parser recursively. This replaces the ad-hoc
parsing used in previous versions, and allows better syntax checking and
catches syntax errors much earlier. The shell attempts to do a much better
job of parsing and expanding array subscripts only once; this has visible
effects in the `unset' builtin, word expansions, conditional commands, and
other builtins that can assign variable values as a side effect. The `unset'
builtin allows a subscript of `@' or `*' to unset a key with that value for
associative arrays instead of unsetting the entire array (which you can still
do with `unset arrayname'). There is a new shell option, `patsub_replacement'.
When enabled, a `&' in the replacement string of the pattern substitution
expansion is replaced by the portion of the string that matched the pattern.
Backslash will escape the `&' and insert a literal `&'. This option is enabled
by default. Bash suppresses forking in several additional cases, including
most uses of $(<file).

All the new features are described below.

Readline has new features as well. There is a new option:
`enable-active-region'. This separates control of the active region and
bracketed-paste. It has the same default value as bracketed-paste, and
enabling bracketed paste enables the active region. Users can now turn off
the active region while leaving bracketed paste enabled. Two new bindable
string variables are available; their values are terminal escape sequences
that set the color used to display the active region and turn it off,
respectively. If set, these are used in place of terminal standout mode.
Finally, Readline now checks for changes to locale settings (LC_ALL/LC_CTYPE/
LANG) each time it is called, and modifies the appropriate locale-specific
display and key binding variables when the locale changes.

There are a few incompatible changes between bash-5.1 and bash-5.2. Here-
documents and here-strings use temporary files if the shell compatibility
level is 50 or lower. The `unset' builtin in bash-5.2 treats array subscripts
`@' and `*' differently than previous versions, and differently depending on
whether the array is indexed or associative. Bash-5.2 attempts to prevent
double-expansion of array subscripts under certain circumstances, especially
arithmetic evaluation, by acting as if the `assoc_expand_once' shell option
were set. Set the compatibility level appropriately to revert to previous
behavior; details are in the file COMPAT.

Bash can be linked against an already-installed Readline library rather
than the private version in lib/readline if desired. Only readline-8.1 and
later versions are able to provide all of the symbols that bash-5.2 requires;
earlier versions of the Readline library will not work correctly.

A complete list of changes between bash-5.1 and bash-5.2 is available in
the file CHANGES; the complete list is too large to include in this
message.

Change in the command substitution parsing code. Is this what's causing the two lines to fail?

[Snowhog]

SNAPLIST1=($(ls /mnt/snapshots/|grep 'ROOT'))
SNAPLIST2=($(ls /mnt/snapshots/|grep 'HOME'))

These lines are now erroring on 23.10. Well, the first one; line 89; which throws me out of the script; the second line isn't executed, but it too, would error, as it's written with the same syntax.

./snapshot.sh: 89: Syntax error: "(" unexpected

I haven't attempted to run this script since installing Kubuntu 23.10 on the laptop. The script hasn't been modified since I finished and verified that it worked; flawlessly. The only thing different is 23.10 vs the prior version that was on the laptop.

Why is this syntax not liked by bash now?

[GreyGeek]

Thank you, and I do (take it as a compliment).

My father once told me that when person A is communicating anything to person B, if person B isn't comprehending, or misunderstands A, the fault isn't with B; A is always responsible for 'effective communication'. I believe that, and do my level best to put it into daily practice.

The only advice I got from my father was "If you get thrown in jail don't call me". I was around 14 or 15 at the time. That was the sum total of his lessons in life. But, I regret not going to his funeral. His journey though life was full of hard knocks not all of his own making.

In six days, I will have been a member of KFN for 15 years!

I'm 13 days beyond my 13th year! (Feb 4, 2009)

[Snowhog]

After readings your posts for the last 13 years I believe you, and I say that as a compliment!

Thank you, and I do (take it as a compliment).

My father once told me that when person A is communicating anything to person B, if person B isn't comprehending, or misunderstands A, the fault isn't with B; A is always responsible for 'effective communication'. I believe that, and do my level best to put it into daily practice.

based on that piece of code I stole from you!

I won't tell!

Can you believe it's been 13 years!

In six days, I will have been a member of KFN for 15 years!

[GreyGeek]

GreyGeek Thank you. My mind operates differently than I believe most peoples do.

After readings your posts for the last 13 years I believe you, and I say that as a compliment! Can you believe it's been 13 years!

[USER="9548"]
As my laptop really doesn't have much more the the installed OS; very little has been added by me (/home/paul); processing the root and home sub-volumes took just little over 14-minutes (the initial 'full backups'). Verifying the script, I waited for some time to elapse then ran it again (Incremental Snapshots). Those only took under 1-minute, as you'd expect.

I probably have several GB more than you in my system, but my long times are around 16 minutes and my inc times are about 30 seconds, based on that piece of code I stole from you!

[Snowhog]

GreyGeek Thank you. My mind operates differently than I believe most peoples do.

As my laptop really doesn't have much more the the installed OS; very little has been added by me (/home/paul); processing the root and home sub-volumes took just little over 14-minutes (the initial 'full backups'). Verifying the script, I waited for some time to elapse then ran it again (Incremental Snapshots). Those only took under 1-minute, as you'd expect.