Announcement

Collapse
No announcement yet.

Actual system backup to bootable USB-Drive

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [DESKTOP] Actual system backup to bootable USB-Drive

    I've been researching this, ever since I waded into the Linux water lake. 90-95% of what I find is the "create a boot-able USB with xx.yy Version and Boot to INSTALL. Youtube, giggle, even in this forum.
    PS> Reminds me. I was reading through Kubuntu.org and they actually have a reference to this forum. Considering how little activity there is (compared to other forums) I was just curious there isn't more users and questions. Maybe it's just me, as I usually don't read everything. My damaged brain can only take so much before bad stuff happens.
    ANYHOW
    I've found a reference to BTRFS and backup (yep, forgot to select this option when I rebuilt the OS a few days ago....see
    [HTML]https://www.kubuntuforums.net/forum/currently-supported-releases/kubuntu-22-04-jammy-jellyfish/post-installation-ax/674746-wont-boot-if-home-drive-connected[/HTML]
    [HTML]https://www.kubuntuforums.net/forum/currently-supported-releases/kubuntu-22-04-jammy-jellyfish/software-support-be/674372-unable-to-upgrade-to-lunar[/HTML]

    BTRFS Reference
    [HTML]https://www.kubuntuforums.net/forum/currently-supported-releases/kubuntu-21-10-impish-indri/software-support-ab/657829-need-help-finding-an-image-backup-app[/HTML]

    AND an external site [HTML]https://askubuntu.com/questions/1260590/creating-a-bootable-usb-with-saved-configurations​[/HTML]

    SO.... EXT4 boot OS on drive SDA1 and /Home on SDB1..

    Just looking for GUI program to run (or script when I plug in the ext USB){even on shutdown} that backups current settings, SELECTED /Home Folders (I have vm's that I don't need backing up) and any OTHER Custom System Settings or changes (wallpapers, windows size & position on opening), etc.
    Encrypting is optional, as if I have to restore, I would try to restore what is broken, now the whole bloody thing, BUTT... depends.

    FYI. I do currently use Lucky Backup, daily (essential /home folders (Firefox, Thunderbird, recently downloaded stuff), and a monthly (Music, vids, VM's)

    Q- Do I really need to backup anything else.... ALSO This could be a guide for other newbies.
    PLEASE
    Include a "HOW TO" Restore on the backup option you tell us about. It's nice to know "click this, select destination, and GO" for a backup, but when things go South, what's the Restore guide

    Many Thanks to Those who leave a valid set of instructions. I know the GreyGeek is going to give us is rundown on the benefits of BTRFS and Snapshots. I still haven't read how to restore a snapshot. No offense GreyGeek. BTRFS sounds like a great file format, and as I have about 80gig spare on my boot drive, I might do a dual boot using BTRFS, just to check it out.

    #2
    You do need to make sure that your ~/.config and ~/.local are backed up, at the very least. You may see other hidden things in your $HOME that you might want to add, if you want.
    All your application and desktop settings are kept in those directories.
    • Install fresh OS, if that is necessary.
    • Install backup Software
    • point it at the location of backup files.
    • restore things as per your specific app's documentation. this one seems to have a wizard to follow?
    • make sure the files are going to the exact same directories that they were saved from.
    • fin! (and probably log out afterwards)
    never used this software myself. It may be quite old, but still seems functional for its purpose.
    Everyone's use cases and needs are different, so I suggest some trial restorations, either from a test account, or by a small unique backup set just for experimenting with.

    I use Kup-backup after trying out numerous tools over the years. I am not recommending it (or not), but I like its feature-set and how it works, and it does what I want with now fuss, unless I muck around of course.
    I also build it from source (which may be why it mostly Just Works for me)
    I have 4 different Plans, some are straight syncing, some are incremental, and there is some overlap, because I am sloppy, and have plenty of space on my NAS
    (I also sync some things with my Nextcloud instance on that NAS, but that is more for remote access than backup)


    While BTRFS is a handy file system for recovery and Oops!! moments, it isn't really a backup tool - the data is on the same file system.
    Having said that, it also has a cool benefit of being able to go back and see previous versions of a file, from when a snapshot was taken. Sort of like incremental backups you probably have on your external drive. Just potentially a lot faster to access and load, since they are right there. GUI tools like Timeshift make this more user-friendly.

    It is a good idea to have both a backup system as well as something like a BTRFS file system, though the latter isn't mandatory. Just very very convenient and useful in the long run.

    Comment


      #3
      I've tried all the BTRFS GUI apps and prefer to use the console and my own bash scripts. Also, I modified my BTRFS installation by moving @home to /home, so that I need only to snapshot @ROOTFS (on my system /dev/sda3) to backup my entire system. In my method I created /mnt onto which I mount my @ROOTFS (the SSD on which my BTRFS resides) and /mnt/snapshots, in which I save my ro snapshots.

      Also, sending a snapshot to an external subvolume needs to be done only once. After that use the incremental backup command. It can reduce backing up time by a factor of 60 or more.

      Here are the commands I use.

      Making a backup snapshot:
      Open a root Konsole, then
      mount /dev/disk/by-uuid/xxxxxxxxxxxxxxx /backup/ (If remote storage is needed)
      mount /dev/disk/by-uuid/xxxxxxxxxxxxxxy /mnt
      btrfs su snapshot -r /mnt/@ mnt/snapshots/@YYYYMMDD (read only. Can't move rw snapshots out of subvol)
      btrfs su snapshot -r /mnt/@home mnt/snapshots/@homeYYYYMMDD (Jump to “Exit” if done)
      (Use blkid command to get uuid to replace xxxxxxxxxxxxxxxxxxxxx)

      Sending snapshots to a remote storage device
      (Slow method but needs to be done only once)
      btrfs send /mnt/snapshots/@YYYYMMDD | btrfs receive /backup/

      sync
      btrfs send /mnt/snapshots/@homeYYYYMMDD | btrfs receive /backup/
      sync

      Making an Incremental Backup to a remote storage device
      (Much, much faster. My 100Gb rootfs takes about an hour without incremental backup. With it the time is under a minute)
      btrfs send -p /mnt/snapshots/@apreviousdate /mnt/snapshots/@todaysdate | btrfs receive /backup

      sync
      btrfs send -p /mnt/snapshots/@homeapreviousdate /mnt/snapshots/@hometodaysdate | btrfs receive /backup
      sync

      Unmount backup and mnt
      umount /backup (If backup was mounted)
      umount /mnt

      Rolling back to a specific snapshot:
      sudo -i
      mount /dev/disk/by-uuid/xxxxxxxxxxxxxxxxxx /mnt
      mv /mnt/@ mnt/@old (the system will continue to use @ as @old)
      sync
      sync
      mv /mnt/@home mnt/@homeold (the system will continue to use @home as @homeold)
      sync
      sync

      (notice the lack of the “-r” parameter in the command below. This allows rw)

      btrfs subvol snapshot /mnt/snapshots/@YYYYMMDD /mnt/@ (this copies the snapshot onto @)
      sync
      btrfs subvol snapshot /mnt/snapshots/@homeYYYYMMDD /mnt/@home
      sync
      umount /backup
      umount /mnt

      Exit root and Konsole (and reboot if @ and/or @home was rolled back)
      exit, exit

      Reboot the computer and delete old subvolumes (If rolling back)
      If reboot is successful then mount /mnt and:
      btrfs subvol delete -C /mnt/@old
      btrfs subvol delete -C /mnt/@homeold

      Read only snapshots can be converted to read-write using

      btrfs property set -ts nameofsnapshot ro false

      and back to read only

      btrfs property set -ts nameofsnapshot ro true
      "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
      – John F. Kennedy, February 26, 1962.

      Comment


        #4
        Originally posted by CharlieDaves View Post
        PS> Reminds me. I was reading through Kubuntu.org and they actually have a reference to this forum. Considering how little activity there is (compared to other forums) I was just curious there isn't more users and questions.
        https://discuss.kde.org is active, a dozen or more new "topics" a day. Actual KDE developers have been "seen" there. Perhaps less friendly than here at KFN, but that's a very high bar IMO.
        Regards, John Little

        Comment


          #5
          Originally posted by jlittle View Post
          Perhaps less friendly than here at KFN, but that's a very high bar IMO.
          Since the move to Discourse, they have improved traffic quite a bit, and imo it even more friendly.

          As another option, there is always Reddit, as long as you mainly stick to r/KDE (quite decent traffic) and r/Kubuntu (very low volume)
          Neither are anything like the typical free-for-all madness or the over-moderated mess that this website can have.
          KDE Devs are seen there as well.

          Originally posted by CharlieDaves View Post
          Considering how little activity there is (compared to other forums) I was just curious there isn't more users and questions.
          Some of this is from the classic forum layouts and motifs are no longer preferred by a LOT of people nowadays. Sort of like mailing lists have mostly died, and IRC is sort of slowing down. Replaced by things that get more traffic today, such as Reddit, and (eeew) Discord servers run by 12 year olds.

          Another factor is that Kubuntu is Ubuntu with Plasma instead of Gnome. Outside of the pointy-clicky bits, everything else is identical. Kubuntu is not a separate distro at all. All the systems and tools under the skin are the same. Sort of like the difference between a Chevy Malibu and a Buick Regal. Same car under the bodywork and interior (bad analogy, I know, but close enough). Users don't need so much kubuntu-specific knowledge, as most of that is either identical to Ubuntu, or general KDE things that are not usually distro-specific.

          Comment


            #6
            Thanks to GreyGeek for new-to-btrfs details.

            A general comment... those who don't do incremental backups that preserve a history of changes to files seem to me to be unaware of what they're missing out on, and think only in terms of restoring to a backup.

            What if important data are deleted or corrupted, then you discover the situation a month or more later, maybe after the full cycle of backup volumes? There's many other cases for which overwriting backups are little or no help. I've found it useful being able to see how a file has changed over the last year or two in a diff tool (side by side, it helped having several monitors). For my desktop using dolphin I can see files from 2018, each month, and weekly (when I've been diligent) from 2021.

            One gets this almost incidentally with btrfs incremental backups, but (I suppose) rsync-based backup tools can do this well if the files that change are not too large. Some management is required to stop churning data, like browser caches, from filling up backup volumes.
            Regards, John Little

            Comment


              #7
              Originally posted by claydoh View Post
              Some of this is from the classic forum layouts and motifs are no longer preferred by a LOT of people nowadays. Sort of like mailing lists have mostly died, and IRC is sort of slowing down. Replaced by things that get more traffic today, such as Reddit, and (eeew) Discord servers run by 12 year olds.
              I thought they were old versions of BOTS kicked out of the Matrix(s) versions LOL

              Comment


                #8
                Originally posted by claydoh View Post
                You do need to make sure that your ~/.config and ~/.local are backed up, at the very least. You may see other hidden things in your $HOME that you might want to add, if you want.
                All your application and desktop settings are kept in those directories.
                I'll add these two directories (folders) into my daily lucky Backup setup.
                Thanks for the suggestions.


                Thanks to all, including GreyGeek, whom I new couldn't resist a challenge at how "geek" he is.
                Ooo. Just noticed that I am now a Veteran member. Am I entitled to a seniors discount? 5% of zero is about ?

                Comment


                  #9
                  Originally posted by jlittle View Post
                  Thanks to GreyGeek for new-to-btrfs details.

                  A general comment... those who don't do incremental backups that preserve a history of changes to files seem to me to be unaware of what they're missing out on, and think only in terms of restoring to a backup.

                  What if important data are deleted or corrupted, then you discover the situation a month or more later, maybe after the full cycle of backup volumes? There's many other cases for which overwriting backups are little or no help. I've found it useful being able to see how a file has changed over the last year or two in a diff tool (side by side, it helped having several monitors). For my desktop using dolphin I can see files from 2018, each month, and weekly (when I've been diligent) from 2021.

                  One gets this almost incidentally with BTRFS incremental backups, but (I suppose) rsync-based backup tools can do this well if the files that change are not too large. Some management is required to stop churning data, like browser caches, from filling up backup volumes.
                  My backup paradigm includes a naming convention: @yyyymmddhhmm. The bash script I use to back up to /mnt/snapshots and to my 500GB Crucial drive via USB stores a maximum of seven snapshots and when the eighth is added the oldest one is deleted.
                  My second script allows me to mount either my internal 1TB m.2 NvME SSD or the Crucial.
                  My 1TB m2 931GB drive has 280GB allotted and 271 Gb used. That 270 Gb contains 19 snapshots of my @rootfs. 80% of that drive will hold 52 snapshots of @rootfs. That's the power of compression that BTRFS uses automatically.

                  That gives me the ability to go back a couple of weeks or more to retrieve something. I also have an HD spinner that I used four times a year to store a copy of @rootfs. This can take me back two or three years, it it also contains copies of files I want to keep permanently.

                  The last thing I do, but never have talked about it, is to keep a rw copy of @rootfs, my most recent being labeled @rwrootfs202310252136, strangely enough. RO copies of snapshots are frozen and are not changed by the BTRFS. RO snapshots are the only kind you can move from one subvolume to another on another, or to a subvolume on a remote drive for archival purposes.
                  IF you open a ro snapshot that you made a week ago it will still be an exact copy of your system as it was when you made that snapshot, even if you've subsequently added or removed many packages and files to your working system.
                  Making a rw snapshot by not using the "-r" flag, which is what @rwrootfs202310252136 is, will result in a subvolume that reflects changes made to your system using the classic cow paradigm. On Debian 12.0 my printer configured automatically without installing HPLIP and also recognized its duplex capability. When I upgraded to Debian 12.1 my printer wouldn't configure or connect, even when using HPLIP and/cups. I later read that HPLIP won't work with 12.1 yet. Meanwhile, I had installed and removed HPLIP, but it left a bunch of detritus in my Downloads subdirectory, which I deleted. In the following image you can see my current system, @rootfs, in the left panel, and the current state of @rwrootfs202310252136 in the right panel.
                  Click image for larger version

Name:	rootfs_compare_20231029.png
Views:	92
Size:	221.4 KB
ID:	674858
                  You can see the extra folders, like hplip-3.23.8, io, installer, data, doc, and others in the right panel. None of it is in the left panel, which shows my current system. I took the 10/25 and @rwrootfs snapshots just before I attempted to remove HPLIP and related software, in case something important got dragged down the toilet with it. Subsequent snapshots do not have HPLIP or its components, but @rwrootfs continues to show them because after deleting them they are no longer part of my working system. If I keep using the 10/25 @rwrootfs and attempt to install HPLIP the changes in each of those components will reflect the change, just as cow promises.

                  .​
                  "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                  – John F. Kennedy, February 26, 1962.

                  Comment


                    #10
                    Originally posted by GreyGeek View Post

                    My backup paradigm includes a naming convention: @yyyymmddhhmm. The bash script I use to back up to /mnt/snapshots and to my 500GB Crucial drive via USB stores a maximum of seven snapshots and when the eighth is added the oldest one is deleted.
                    My second script allows me to mount either my internal 1TB m.2 NvME SSD or the Crucial.
                    My 1TB m2 931GB drive has 280GB allotted and 271 Gb used. That 270 Gb contains 19 snapshots of my @rootfs. 80% of that drive will hold 52 snapshots of @rootfs. That's the power of compression that BTRFS uses automatically.

                    That gives me the ability to go back a couple of weeks or more to retrieve something. I also have an HD spinner that I used four times a year to store a copy of @rootfs. This can take me back two or three years, it it also contains copies of files I want to keep permanently.

                    The last thing I do, but never have talked about it, is to keep a rw copy of @rootfs, my most recent being labeled @rwrootfs202310252136, strangely enough. RO copies of snapshots are frozen and are not changed by the BTRFS. RO snapshots are the only kind you can move from one subvolume to another on another, or to a subvolume on a remote drive for archival purposes.
                    IF you open a ro snapshot that you made a week ago it will still be an exact copy of your system as it was when you made that snapshot, even if you've subsequently added or removed many packages and files to your working system.
                    Making a rw snapshot by not using the "-r" flag, which is what @rwrootfs202310252136 is, will result in a subvolume that reflects changes made to your system using the classic cow paradigm. On Debian 12.0 my printer configured automatically without installing HPLIP and also recognized its duplex capability. When I upgraded to Debian 12.1 my printer wouldn't configure or connect, even when using HPLIP and/cups. I later read that HPLIP won't work with 12.1 yet. Meanwhile, I had installed and removed HPLIP, but it left a bunch of detritus in my Downloads subdirectory, which I deleted. In the following image you can see my current system, @rootfs, in the left panel, and the current state of @rwrootfs202310252136 in the right panel.
                    Click image for larger version

Name:	rootfs_compare_20231029.png
Views:	92
Size:	221.4 KB
ID:	674858
                    You can see the extra folders, like hplip-3.23.8, io, installer, data, doc, and others in the right panel. None of it is in the left panel, which shows my current system. I took the 10/25 and @rwrootfs snapshots just before I attempted to remove HPLIP and related software, in case something important got dragged down the toilet with it. Subsequent snapshots do not have HPLIP or its components, but @rwrootfs continues to show them because after deleting them they are no longer part of my working system. If I keep using the 10/25 @rwrootfs and attempt to install HPLIP the changes in each of those components will reflect the change, just as cow promises.

                    .​

                    PS - I forgot to add that since @rwrootfs is a rw subvolume it will continue to grow as I use my system, taking up more and more disk space. Eventually it will eat up enough space that I will begin to notice my BTRFS system slowing down. Before that point is reached it is a good idea to delete the old @rwrootfsyyyymmddhhmm subvolume with a new one.
                    "A nation that is afraid to let its people judge the truth and falsehood in an open market is a nation that is afraid of its people.”
                    – John F. Kennedy, February 26, 1962.

                    Comment

                    Working...
                    X