[SOLVED] Need help setting up multi-boot with btrfs

[TwoFistedJustice]

I've begun a thorough step by step write up for EFI. For now it's on github and mostly unformatted. It's quite long and I still have a ways to go.

Feel free to comment either here or there, whichever suits your methods better.

The public github repo is here: Better Multi-Booting with BTRFS and Linux

Does this forum have the ability to support posts in markdown?

[TwoFistedJustice]

A good idea, but that's a commitment...

IMO it would actually be pretty easy to write a quality how-to that way because github has excellent feedback tools which make suggestions easier to track and implement than a forum thread does. Then there's the advantages git itself provides of branching and reversion.

[jlittle]

... a public github repo ...

A good idea, but that's a commitment...

It also has the advantage of being easily accessed by people using other distros outside our little Kububble.

In the past Google has been very good at sending searchers in to kubuntuforums.net, not on the scale of the Arch wiki but similar. I wonder if that is still the case.

[TwoFistedJustice]

You should combine your posts into one NEW contiguous step-wise explanation on how to set up a multi-boot system from scratch using what you've learned.
oshunluvr could then make it sticky.

If oshunluvr agrees I'll be happy to do the write up.

Given the complexity of the process (even though it's not actually THAT hard to do once you get into it ) I would suggest that rather than starting with a new thread which has a long string of corrections and commentary beneath it, as is likely to be the case, I begin it with a markdown file in a public github repo and we manage it from there. Then once it's satisfactory port it over here and make it a sticky. That way the space beneath the sticky is open for questions from those who come after. Then they don't have to dig through multiple pages of sub-posts to get help.

It also has the advantage of being easily accessed by people using other distros outside our little Kububble.

Thoughts?

[TwoFistedJustice]

Cool, good to know. I suspect that's not using a symlink but relying on grub to find something to boot; maybe if there are several kernels present you might not get the most recent.

Different distros use different names, and Ubuntu changed the location and name some years ago. For example, when I last installed Gentoo it called the linux executable "kernel" and the initial RAM disc "initramfs". If booting an Ubuntu from an iso the linux is still "vmlinuz" but the initial RAM disc is just "initrd". One can always look in the boot directory (mounting from a live USB) to see what's there.

There are several projects that compile how to boot distros, including Ventoy.

I did a little file renaming to see how Kali would react. On my Kali system I have two kernels: 6.1 & 6.3. I check the kernel version with 'uname -r'

With the * syntax it chose 6.3 over 6.1

I thought it might just be globbing and since the file names are identical up to the second digit it simply chose based on ascii ordering.

I copied all the 6.1 files and renamed them by inserting an "A". So instead of "vmlinuz-6.1 ..." it changed to "vmlinuzA-6.1 ..."

When I rebooted, Kali started up fine and the running kernel version was 6.3. So it ignored my renamed file and chose the latest version.

Then I considered that it might be looking inside the file and getting the version number that way. So I renamed my "A" files like "vmlinuz-6.4" (these were actually 6.1). After restart the kernel version showed as 6.1.

So it must have chosen based on the version in the file name.

[jlittle]

I tried this out with Kali and it doesn't (quite) work. Kali needs a wildcard character tacked to the end. i.e.

linux /@kali/boot/vmlinuz*
initrd /@kali//boot/initrd.img*

Cool, good to know. I suspect that's not using a symlink but relying on grub to find something to boot; maybe if there are several kernels present you might not get the most recent.

Different distros use different names, and Ubuntu changed the location and name some years ago. For example, when I last installed Gentoo it called the linux executable "kernel" and the initial RAM disc "initramfs". If booting an Ubuntu from an iso the linux is still "vmlinuz" but the initial RAM disc is just "initrd". One can always look in the boot directory (mounting from a live USB) to see what's there.

There are several projects that compile how to boot distros, including Ventoy.

[GreyGeek]

I now have Kubuntu and Kali dual booting with EFI from the same btrfs partition.

You should combine your posts into one NEW contiguous step-wise explanation on how to set up a multi-boot system from scratch using what you've learned.
oshunluvr could then make it sticky.

[TwoFistedJustice]

So assuming this is the same in Kali as *buntu, you should use this instead of the above in your 40_custom:

linux /@kali/boot/vmlinuz
initrd /@kali//boot/initrd.img

I tried this out with Kali and it doesn't (quite) work. Kali needs a wildcard character tacked to the end. i.e.

linux /@kali/boot/vmlinuz*
initrd /@kali//boot/initrd.img*

After setting this up I updated from 6.1 to 6.3, ran uname -r and confirmed that it used 6.3.


[EDIT]

Contents of @kali/boot


drwxr-xr-x 1 root root 644 Jul 14 14:26 .
drwxr-xr-x 1 root root 258 Jul 13 15:43 ..
-rw-r--r-- 1 root root 258800 May 12 11:20 config-6.1.0-kali9-amd64
-rw-r--r-- 1 root root 262471 Jun 29 05:05 config-6.3.0-kali1-amd64
-rw-r--r-- 1 root root 258800 Jul 14 14:11 config-6.4.0-kali1-amd64
drwx------ 3 root root 4096 Dec 31 1969 efi
drwxr-xr-x 1 root root 106 Jul 13 15:43 grub
-rw-r--r-- 1 root root 76430800 Jul 8 18:37 initrd.img-6.1.0-kali9-amd64
-rw-r--r-- 1 root root 79739435 Jul 13 15:44 initrd.img-6.3.0-kali1-amd64
-rw-r--r-- 1 root root 76430800 Jul 14 14:11 initrd.img-6.4.0-kali1-amd64
-rw-r--r-- 1 root root 83 May 12 11:20 System.map-6.1.0-kali9-amd64
-rw-r--r-- 1 root root 83 Jun 29 05:05 System.map-6.3.0-kali1-amd64
-rw-r--r-- 1 root root 83 Jul 14 14:12 System.map-6.4.0-kali1-amd64
-rw-r--r-- 1 root root 7976256 May 12 11:20 vmlinuz-6.1.0-kali9-amd64
-rw-r--r-- 1 root root 8983392 Jun 29 05:05 vmlinuz-6.3.0-kali1-amd64
-rw-r--r-- 1 root root 7976256 Jul 14 14:11 vmlinuz-6.4.0-kali1-amd64

[oshunluvr]

linux /@/boot/vmlinuz-6.1.0-kali9-amd64

I think this was mentioned above, but to reiterate; using a 40_custom entry with the kernel version like this will eventually lead to either not being able to boot to this distro (if this specific kernel is removed) or require editing of 40_custom with every kernel update to amend the version number.

I'm not sure about Kali, but *buntu distros also maintain links to the current and previous kernels as you can see here:

stuart@office:/boot$ ll
total 407M
drwxr-xr-x 1 root root  826 Jul 13 13:17 ./
drwxr-xr-x 1 root root  178 Jul 11 07:37 ../
-rw-r--r-- 1 root root 264K Jun  7 10:23 config-5.19.0-45-generic
-rw-r--r-- 1 root root 264K Jun 21 10:38 config-5.19.0-46-generic
drwxr-xr-x 1 root root  132 Jul  1 08:54 grub/
lrwxrwxrwx 1 root root   28 Jun 29 15:55 initrd.img -> initrd.img-5.19.0-46-generic
-rw------- 1 root root 122M Jul  3 09:23 initrd.img-5.19.0-45-generic
-rw------- 1 root root 122M Jul 11 11:33 initrd.img-5.19.0-46-generic
lrwxrwxrwx 1 root root   28 Jul 13 13:17 initrd.img.old -> initrd.img-5.19.0-45-generic
-rw------- 1 root root 6.2M Jun  7 10:23 System.map-5.19.0-45-generic
-rw------- 1 root root 6.2M Jun 21 10:38 System.map-5.19.0-46-generic
lrwxrwxrwx 1 root root   25 Jun 29 15:00 vmlinuz -> vmlinuz-5.19.0-46-generic
-rw------- 1 root root  12M Jun  7 10:23 vmlinuz-5.19.0-45-generic
-rw------- 1 root root  12M Jun 21 10:43 vmlinuz-5.19.0-46-generic
lrwxrwxrwx 1 root root   25 Jul 13 13:16 vmlinuz.old -> vmlinuz-5.19.0-45-generic
​

"vmlinuz" points to the latest kernel
"vmlinuz.old" points to the previous kernel
and same for initrd.img.

So assuming this is the same in Kali as *buntu, you should use this instead of the above in your 40_custom:

linux /@kali/boot/vmlinuz
initrd /@kali//boot/initrd.img

Alternately, I use this in 40_custom:

menuentry 'Kali GNU/Linux' --class kali {
insmod part_gpt
insmod btrfs
search --no-floppy --fs-uuid --set=root 73f6525c-0f9c-4a23-a91b-d1b46f5079c8
configfile /@kali/boot/grub/grub.cfg
}

Then what you get is nested GRUB menus. If you select "Kali" from your boot menu, you would get a second GRUB menu. Then you can just press Enter or select from the boot menu normally. Pressing "ESC" returns you to the first boot menu. I have 4-5 bootable distros at times and a couple ISOs that I can boot to in this way. I set my GRUB timeouts at 3 secs so the delay isn't significant if I don't press Enter.

[TwoFistedJustice]

I now have Kubuntu and Kali dual booting with EFI from the same btrfs partition.

This is the final step after post #22:

Prerequisites and Early Steps:

You've got Kubuntu installed and the subvolumes renamed per above and everything works.

You have a bootable Linux USB stick so you can access the file system of your new distro from outside.

Whenever you need to update grub by run 'sudo grub-mkconfig' then 'sudo update-grub'.

Now you install your second distro to the btrfs partition.

Verify that your new distro works, you should once again have @ and @home subvolumes, plus any others the new distro puts on there. (Kali adds a BUNCH)

Set your grub ownership by booting into your chosen distro and updating grub.


How to Configure The Next Distro

There is one new step. And one omission.

You will NOT need to alter the $prefix variable on the secondary distros because your primary distro controls the boot process and its $prefix should already be altered (post #22).


You will need to find /boot/grub/grub.cfg in the new distro.

Down in the section

 ### BEGIN /etc/grub.d/10_linux ###

you will find and COPY a menu entry that looks something like this:

menuentry 'Kali GNU/Linux' --class kali --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-73f6525c-0f9c-4a23-a91b-d1b46f5079c8' {
load_video
insmod gzio
if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
insmod part_gpt
insmod btrfs
set root='hd0,gpt3'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt3 --hint-efi=hd0,gpt3 --hint-baremetal=ahci0,gpt3 73f6525c-0f9c-4a23-a91b-d1b46f5079c8
else
search --no-floppy --fs-uuid --set=root 73f6525c-0f9c-4a23-a91b-d1b46f5079c8
fi
echo 'Loading Linux 6.1.0-kali9-amd64 ...'
linux /@/boot/vmlinuz-6.1.0-kali9-amd64 root=UUID=73f6525c-0f9c-4a23-a91b-d1b46f5079c8 ro rootflags=subvol=@ quiet splash
echo 'Loading initial ramdisk ...'
initrd /@/boot/initrd.img-6.1.0-kali9-amd64
}​

Extract that by whatever means you like to your main distro. ( I pasted it into a text file and moved it via usb stick)

Change the three "@" subvolume names down at the bottom to whatever you intend to name your new distro @ subvolume.

[edit] See the next two posts (28, 29) for a syntax refinement for the last two lines in the menu entry. [/edit]

Paste it into the file /etc/grub.d/40_custom in your main distro.

Do everything else for the new distro using the same methods as you did for Kubuntu.

Then update grub in your main distro and you should be golden, or purple, or whatever.

[TwoFistedJustice]

LOL, I fiddle with it from time to time. I have a laptop that I left Windows on after I bought it for warranty purposes. I managed to muck up EFI a couple times before I got it doing what I wanted. ...

I've been trying to figure out where Kali sets the prefix variable. Finally figured out it's set in a binary rather than a text file. In my search something reminded me of oshunluvr (bottom of readout screenshot )

[oshunluvr]

Anyway, I installed Kali and it took over grub (as expected). I'm going to fiddle around and see if I can break, err I mean, figure it out on my own. If I need help with that I'll start a new thread.

One thing you can usually do to avoid this it to tell the installer to install GRUB to a partition. Then when you reboot, the previous grub is still on the main part of the drive. Also, not a huge deal to boot into the install you want to control grub from and re-run grub-install.

It really depends on how often you expect to install new OSs on your system. If you're doing it more than a couple time a year it might be worth considering having a dedicated grub OS.

Gee I hope I didn't end your decades long streak of successfully avoiding EFI! :-)

LOL, I fiddle with it from time to time. I have a laptop that I left Windows on after I bought it for warranty purposes. I managed to muck up EFI a couple times before I got it doing what I wanted. Part of the problem was earlier versions of KDEneon and Kubuntu didn't identify themselves separately from ubuntu so you couldn't (or at least I couldn't) get both of them to boot on the same computer. Now-a-days both flavors are clearly separate and Windows still boots too.

[TwoFistedJustice]

Glad you figured it out for yourself. I installed Kubuntu 22.04 into a VM using EFI just so I could check the EFI contents and verify my suspicion.

You beat me to it!

Gee I hope I didn't end your decades long streak of successfully avoiding EFI! :-)

Anyway, I installed Kali and it took over grub (as expected). I'm going to fiddle around and see if I can break, err I mean, figure it out on my own. If I need help with that I'll start a new thread.

Thanks again for you help!

[oshunluvr]

Glad you figured it out for yourself. I installed Kubuntu 22.04 into a VM using EFI just so I could check the EFI contents and verify my suspicion.

You beat me to it!

[TwoFistedJustice]

( THIS IS THE ONE WITH THE ANSWER ) ( -- also see post#27 )

WOO HOO! FIXED IT! Now she boots right up!

As oshunluvr speculated, it is in fact something to do with EFI. There is one more file to edit. And like our friend grub.cfg, it is called... wait for it... wait for it... ( dramatic music ) ... grub.cfg !!!!

It's location is /boot/efi/EFI/ubuntu/grub.cfg

Its contents look like this:

search.fs_uuid 73ffa900-dca2-47f2-9e0b-f8c1942ef918 root hd0,gpt2
set prefix=($root)'/@/boot/grub' ## Change this line
configfile $prefix/grub.cfg
​

in my case, because I named my subvolume "@ku" I changed that line to:

set prefix=($root)'/@ku/boot/grub'

Now there's another part, namely accessing that file. It resides on your EFI partition.

You can use Krusader in root mode.

If you want to access it via a Linux Live doohickey, then you have mount it, as explained by the eminent Rod Smith

I'm going to play with it a little more because when I select 'Kubuntu 22.04' at the grub menu, it just sends me back to the top of the menu and I have to select 'Ubuntu'. I think this has to do with the 40_custom edit. It's a fresh system with absolutely nothing on it. And now that I know how to do it, it will take 5 minutes to replicate.

I have yet to install another distro.

Thanks everyone for your help! Thank you oshunluvr for sharing your cool discovery!