starting sector number, 4532393984 exceeds the msdos-partition-table-imposed maximum

[rwbehne1]

# mkfs.btrfs -f /dev/sdf1
btrfs-progs v4.9.1
See http://btrfs.wiki.kernel.org for more information.

ERROR: failed to check size for /dev/sdf1: No such file or directory

Nothing seems to go smoothly for me.

[GreyGeek]

So, if I were to start over, assuming that I could copy all my home partition to another drive, dos this mean when installing the brtfs I don't make any swap or other partitions?

I have been working on formatting a spare 1T drive and I put btrfs on it. Is it ready at this point for me to just start copying to it? or do I have to create a subvolume first, and how?

You don't need a swap with Btrfs. IF you keep an EXT4 partiton that has a distro installed on it a swap drive would be useful for it.

All you need to do is use KParted and format the entire drive using Btrfs. Open KDE Partition Manager and in the left panel left click on the drive you want to reformat. In the right panel right mouse on the partions(s) and choose delete. Repeat until there is only one unformated partition. Right mouse again on it and chose format. From the options select Btrfs. Commit the changes. What you'll get is an HD formatted with Btrfs but without an operation system. You can add its UUID and mount point to /etc/fstab or you can do what I do: don't mount it until you need it and then umount it when you are through. Here's what my /dev/sdb1 looks like:

# mount /dev/sdb1 /backup/
root@jerry-Aspire-V3-771:~# vdir /backup/
total 0
drwxr-xr-x 1 root root 398 Jul 28 21:06 @_bkup20170725
drwxr-xr-x 1 root root 398 Aug 10 12:45 @_bkup20170810
drwxr-xr-x 1 root root 398 Aug 19 11:10 @_bkup20170819
drwxr-xr-x 1 root root  10 Jul 28 21:49 @home_bkup20170725
drwxr-xr-x 1 root root  10 Aug 10 11:53 @home_bkup20170810
drwxr-xr-x 1 root root  10 Aug 19 12:06 @home_bkup20170819

It is NOT part of tthe /dev/sda1 and /dev/sdc that make up my RAID1 pool. I use it specifically to "send & receive" ro dated snapshots of @ and @home.

I'll need to compress files as efficiently as possible in hopes of fitting it all on one drive. I'm trying to format a 500G drive too, but it won't take a new partition table for some unknown reason. Should I choose GPT or MS-Dos?

I wish that there were an easy way to convert an existing ext4 to btrfs, resize the partition, and do it without loosing any data.

I've stopped using MSDOS and go with GPT (GUIID Partition Table), which allows working with partitions greater than 2Tb, and creating up to 128, IIRC, primary partitions. The total data storage space of a PC HDD on which MBR partitioning is implemented can contain at most four primary partitions, or alternatively three primary partitions and an extended partition. The extended partition can contain logical drives. A max of four, IIRC.

When formatting a drive to Btrfs you can enable compression, but I don't use it, so I don't have anything valid to say about it.

There is a command that allows you to convert an EXT partition to Btrfs. See: https://btrfs.wiki.kernel.org/index....sion_from_Ext3

Notice the warning.

Warning: As of 4.0 kernels this feature is not often used or well tested anymore, and there have been some reports that the conversion doesn't work reliably. Feel free to try it out, but make sure you have backups.

Make backups and you don't need to use the convert program. Just format the HD and then copy from your backup onto the Btrfs formatted HD. Note: when you mount a drive that is just formatted with Btrfs you will not see a @ or @home subvolume because you didn't install a distro on it. You just formatted it. After formatting it you can treat it like it was an EXT4 by saving and deleting files like you normally do. You can use mkdir to make directories, which are NOT subvolumes, and cannot be converted to subvolumes, but, you can create directories inside subvolumes using mkdir.

https://askubuntu.com/questions/1190...mes-vs-folders
"The difference between a folder and a subvolume is that subvolumes can be mounted without mounting either the parent subvolume (root) or any other subvolume. And subvolumes can be snapshotted, whereas folders can't be."

But, a snapshot of a subvolume will include ALL the folders and subvolumes in it.

[oshunluvr]

The swap comment refers to using a swap file rather than a swap partition. Most of us don't do that. A swap partition make more sense in most cases and works better. Just use a swap partition and don't use swap files - with any file system you choose.

Definitely use GPT over MBR but know that if you're booting to a GPT drive you have to make a partition for EFI or GRUB - whichever you're booting with. I have already done a How-To on making a GRUB partition on a GPT drive here.

You can easily convert a partition from EXT4 to btrfs, but it doesn't put the files into a subvolume for you. You would have to make a subvolume yourself and move the files into it - not a huge task. BTRFS supports compression as a mount option as well as auto-defrag and others. Resizing is also possible, but IME regardless of file system resizing not an easy or quick task and sometimes ends in disaster. This is one of the big advantages of BTRFS - no partitions to resize.

If you're moving your home from the current EXT4 drive/partition onto a third drive, yes make a subvolume first. Sorry, but virtually all btrfs functions at this level require terminal commands. You make a subvolume then mount it just like you would a partition.

I will assume the third drive is sdc and you have partitioned it with a single partition using the entire drive;

sudo mkfs.btrfs /dev/sdc1
sudo mkdir /mnt/newhome
sudo mount /dev/sdc1 -o autodefrag,space_cache,compress /mnt/newhome 
sudo btrfs su cr /mnt/newhome/@newhome
sudo umount /dev/sdc1
sudo mount /dev/sdc1 -o autodefrag,space_cache,compress,subvol=@newhome /mnt/newhome

You now have the subvolume /dev/sdc1/@newhome mounted at /mnt/newhome and ready for files.

[rwbehne1]

Be advised that Btrfs does not use a /swap file.

So, if I were to start over, assuming that I could copy all my home partition to another drive, dos this mean when installing the brtfs I don't make any swap or other partitions?

I have been working on formatting a spare 1T drive and I put btrfs on it. Is it ready at this point for me to just start copying to it? or do I have to create a subvolume first, and how?

I'll need to compress files as efficiently as possible in hopes of fitting it all on one drive. I'm trying to format a 500G drive too, but it won't take a new partition table for some unknown reason. Should I choose GPT or MS-Dos?

I wish that there were an easy way to convert an existing ext4 to btrfs, resize the partition, and do it without loosing any data.

[oshunluvr]

Well, hopefully you'll get the boot issue squared away. It's much safer and usually required to do partition level modifications while the device is entirely off-line (not mounted and no partition in use). I did state USB, but I assume you understand that a bootable CD would work as well. I haven't read your other post yet, but I will.

How is partitioning replaced by BTRFS? That's something I really don't understand, but need to.

The short answer is "Subvolumes".

Briefly, btrfs uses subvolumes to segregate data in a way similar to partitioning. However, since the subvolumes all exist on a single file system, the remaining free space is available to all subvolumes. Reading through GreyGeeks post might help you understand

For example, the default behavior of the Kubuntu installer if you select btrfs at install time will create a single btrfs file system on the target partition/drive and then create two subvolumes - one for root, one for home. These subvolumes are individually mountable just like partitions. After install, when you boot for the first time, the "mount" command (which displays all mounted file systems) would reveal something like:

:~$ mount
/dev/sdc3 on / type btrfs (rw,noatime,compress=lzo,ssd,space_cache,autodefrag,subvolid=4454,subvol=/@)
/dev/sdc3 on /home type btrfs (rw,noatime,compress=lzo,ssd,space_cache,autodefrag,subvolid=4455,subvol=/@home)

The /etc/fstab entries for these look like:

UUID=8f0c1661-4e84-4512-b875-23bcfd5be1d8 /              btrfs rw,noatime,space_cache,compress=lzo,autodefrag,subvol=@ 0 1
UUID=8f0c1661-4e84-4512-b875-23bcfd5be1d8 /home          btrfs rw,noatime,space_cache,compress=lzo,autodefrag,subvol=@home 0 2

The default naming scheme is to name the root subvolume "@" and home "@home" as you see above in the options list to the far right. Note the device name - /dev/sdc3 - and the UUID is the same for both of these mounts. They both exist on /dev/sdc3 but as separate subvolumes. If I mount the root of the btrfs file system - in this case /dev/sdc3 - without specifying a specific subvolume the entire subvolume structure is revealed:

:~$ sudo mount /dev/sdc3 /mnt/subvolumes
:~$ ls -l /mnt/subvolumes
drwxr-xr-x 1 root root 286 Aug 17 09:52 @/
drwxr-xr-x 1 root root  38 Apr  1 09:30 @home/

and df reveals both subvolumes have the same amount of used and free space:

:~$ df 
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdc3       460G  303G  156G  66% /
/dev/sdc3       460G  303G  156G  66% /home

Effectively, the size of any subvolume expands (and contracts) as needed as long as room exists on the file system.

Not knowing exactly why you're splitting up /opt and /var and /usr/local, but lets assume your reasons are valid and you still want to do this. You could, after installation, manually create subvolumes for each of these, move the files from the install subvolume into the new subvolumes, and mount the subvolumes at their respect locations. You would then have separate locations for these files but all five of your subvolumes would be sharing the same space. No wasted space because you miscalculated the needed size for a particular partition. There are many other benefits to btrfs that I won't go into here.

I think the reason I can't glean why you need /var, etc., separately mounted is in my experience, this was typically done to span needed space for an install across multiple devices or to add-in an additional partition to an existing install without resizing existing partitions. Back in the day, I had four small hard drives and having /var on a separate hard drive effectively enlarged the available space for the install. Now that hard drives are so large, this is rarely needed.

[GreyGeek]

.....
One thing you said did confuse me:

I know nothing about BTRFS, yet, and would highly value your "entry-level" explanation as to how that works, since I've already learned more from you in this thread than in over a year in all other forums. My thinking is to have certain partitions which never get formatted, that way my data remains between new installs of the OS. How can you format for a new install if everything is on one BTRFS partition? The only reason I have for multiple partitions is to preserve the data which gets installed in them, and I don't think there's a way of having everything in /home, /usr/local, /opt, and /var in one partition together, while / and everything else are in another partition which can be formatted for clean installs.

How is partitioning replaced by BTRFS? That's something I really don't understand, but need to.
...

Pay careful attention to what oshunluver is saying and follow his recommendations and you'll be entering fs narvana!

Using Btrfs you can create subvolumes which can be treated like partitions. Also, unlike EXT4, you don't have to guesstimate how much disk space you'll need for each "partition" you want to make. Subvolumes take what they need out of the unused pool and no disk space is wasted. Be advised that Btrfs does not use a /swap file.

I switched to Btrfs 2.5 years ago and I have only one word to describe my experience: awesome!
Ubuntu's (Kubuntu, Neon, etc) Btrfs hierarchy is, IMO, superior to, say, SUSE's. Off of the Btrfs "ROOT" you have @ (which is /) and @home (which is /home). On a live, running Btrfs installation you can open a Konsole, switch to root, and issue
mount /dev/sdaX /mnt
where /dev/sdaX is the partition on which Btrfs is installed. Do a directory of /mnt and you'll get, as shown by my setup,

# vdir /mnt
total 0
drwxr-xr-x 1 root root 398 May 12 13:52 @
drwxr-xr-x 1 root root 10 Apr 24 10:32 @home
drwxr-xr-x 1 root root 192 Aug 19 10:56 snapshots

"snapshots" is a subdirectory I made to store subvolume snapshots of @ and @home

# vdir /mnt/snapshots/
total 0
drwxr-xr-x 1 root root 398 May 12 13:52 @_bkup20170801
drwxr-xr-x 1 root root 398 May 12 13:52 @_bkup20170810
drwxr-xr-x 1 root root 398 May 12 13:52 @_bkup20170819
drwxr-xr-x 1 root root 10 Apr 24 10:32 @home_bkup20170801
drwxr-xr-x 1 root root 10 Apr 24 10:32 @home_bkup20170810
drwxr-xr-x 1 root root 10 Apr 24 10:32 @home_bkup20170819

I have three 750Gb HD's in my laptop. Two of them are set up as RAID1, with the system, data and meta-data .

[FONT=monospace][COLOR=#000000]# btrfs filesystem show /[/COLOR]
Label: none  uuid: 12980ae8-4117-4cc5-bbb8-8065e82af93d
        Total devices 2 FS bytes used 130.47GiB
        devid    1 size 691.19GiB used 135.03GiB path /dev/sda1
        devid    2 size 698.64GiB used 135.03GiB path /dev/sdc

or

[/FONT][FONT=monospace][COLOR=#000000]# btrfs filesystem show  [/COLOR]
Label: none  uuid: 12980ae8-4117-4cc5-bbb8-8065e82af93d
        Total devices 2 FS bytes used 130.47GiB
        devid    1 size 691.19GiB used 135.03GiB path /dev/sda1
        devid    2 size 698.64GiB used 135.03GiB path /dev/sdc

Label: none  uuid: 17f4fe91-5cbc-46f6-9577-10aa173ac5f6
        Total devices 1 FS bytes used 327.49GiB
        devid    1 size 698.63GiB used 332.02GiB path /dev/sdb1

[/FONT]

and the syste, data and meta-data:

[FONT=monospace][COLOR=#000000]# btrfs filesystem df /[/COLOR]
Data, RAID1: total=133.00GiB, used=129.25GiB
System, RAID1: total=32.00MiB, used=48.00KiB
Metadata, RAID1: total=2.00GiB, used=1.22GiB
GlobalReserve, single: total=229.42MiB, used=0.00B
[/FONT]

Btrfs RAID1 is self-healing.
The third HD is a Btrfs backup volume, not in the RAID1 pool.

In another recent post I described how easy (3 minutes) it was to roll back to a previous snapshot after I played with a Siri-like speech command tool.

[rwbehne1]

You didn't say (that I saw) if you're keeping the Fedora install or not - I assume you meant you're deleting it if it's on /dev/sda7.

You assume correctly. I don't like Fedora, as I have discovered, so it needs to go away. Quickly. So the swap, /tmp, and root partitions all will be deleted, then, since you said...

you need a small partition using sectors 34-2047 of the type ef02 for grub to use.

I will do that, then create a new swap space, and fill the remaining space between swap and /home with the root partition. That takes care of the unused space at the bottom section of the drive. For the top I need to use that space too, but for something else. Since some add-on programs like to install to /opt by default I want to preserve that data between formats. I don't understand why there's a /srv partition in every Linux, but all servers get put into /var instead. But I do want to either preserve the servers which are in /var, or move them to /srv so as to preserve that data between each system install/upgrade. I see no reason to preserve the other data in the /var partition, so perhaps instead of a /var partition I'd use a /srv partition and relocate the servers.

One thing you said did confuse me:

If it were me, I'd stop using partitions altogether and use BTRFS instead of out-dated EXT formatting

I know nothing about BTRFS, yet, and would highly value your "entry-level" explanation as to how that works, since I've already learned more from you in this thread than in over a year in all other forums. My thinking is to have certain partitions which never get formatted, that way my data remains between new installs of the OS. How can you format for a new install if everything is on one BTRFS partition? The only reason I have for multiple partitions is to preserve the data which gets installed in them, and I don't think there's a way of having everything in /home, /usr/local, /opt, and /var in one partition together, while / and everything else are in another partition which can be formatted for clean installs.

How is partitioning replaced by BTRFS? That's something I really don't understand, but need to.

Lastly, there is one really big problem remaining which is stopping me in my tracks: the fact that the USB drive won't boot - it just hangs! I've made plenty of bootable USB drives in the past and never ran into this sort of problem before. I posted a question about that (SEE: my post,) but haven't gotten even a hint yet, and have been spending my waiting time by googling for a solution, but finding nothing useful. Once I can get this USB drive to boot I'll be eager to follow your instructions.

[oshunluvr]

OK, for starters:

There is no way to be 100% certain that messing with a disk at this level won't accidentally, permanently, remove data. The ONLY way, ever, to protect data is having a backup or backups, period. IF your data is important, you MUST make a backup or there is always a chance it will be gone.

Sorry, but that's just the way it is. You might consider changing your plans until you can make a backup. On the other hand, since you have no backup now you're already at risk - you're just one power surge or hard drive failure away from zero data - so maybe you should just forge ahead. Your call obviously.

Having said that, I've done this sort of thing before without problems. One thing you have going for you is no Windows so you won't have to worry about that.

The steps required are:
1. Converting to GPT
2. Deleting unwanted partitions and making new ones.
3. Making the disk boot-able again.

The conversion will leave gaps in between your current partitions because of the inefficiencies of MBR formatting. Fortunately, your swap partition and tmp partitions are Logical so you can just delete them and re-create them. You didn't say (that I saw) if you're keeping the Fedora install or not - I assume you meant you're deleting it if it's on /dev/sda7. If there are leftover gaps (unused space) between partitions, it will be small. You can decide for yourself if you want to move or expand partitions to fill the space.

I'm not sure what problems having separate /var and /opt (or /usr/local) would solve as almost no one does that anymore. IME this causes more problems than it solves, but whatever, it's your machine. If it were me, I'd stop using partitions altogether and use BTRFS instead of out-dated EXT formatting, but that's a different topic.

Assuming /tmp, swap, and Fedora are all the logical partitions and you're deleting them, here's how I would proceed:

Boot to a LiveUSB of Kubuntu, open a terminal, do sudo gdisk /dev/sdb (or whatever device your hard drive is after booting to the USB). This will result in a warning that if you continue, you will be converting to GPT. At this point, simply continuing and writing the new table to disk will complete that conversion. Then delete the unneeded partitions and create your new ones.

Since you're doing a new install to this disk, making the disk bootable is rather easy. Short version is you need a small partition using sectors 34-2047 of the type ef02 for grub to use. Here's a link to a longer explanation. Once you have this partition, install Kubuntu and grub should install just fine.

gdisk is menu driven and easy to use. All commands are single letters and entering a ? will print a list of available commands anytime you need them. If you need help using this utility, post back and I'll walk you through it.

[rwbehne1]

I forgot to mention, the version I'll be installing is kubuntu-17.04-desktop-amd64.iso.

You have to switch to GPT formatting.

I am not at all familiar with GPT formatting, whatever that is. Can it be switched to without touching my home and /usr/local partitions? I have no way to back up those partitions at this time.

[break]

Ok, oshunluvr, I've been googling and found "GPT (GUID Partition Table)". Now I'm starting to grock it, albeit slowly. It's a choice done during install, and I've always selected MBR out of habit. So I have to switch to installing the boot stuff to the GUID Partition Table instead. That shouldn't be a problem. Are there any caveat emptors I should know about in advance?

[rwbehne1]

This is because you're using MBR and it only supports 2.2TB or 2TiB. You have to switch to GPT formatting.

Questions:
1) Are you booting to this device?
2) If #1 is "yes" are you using GRUB or EFI to boot?
3) What does your current partition table look like? We need a detailed print out. The results of sudo fdisk -l /dev/sda should do it.
4) What version of Kubuntu are you using?

Answer these and we can help you convert.

1) Yes
2) GRUB
3)

# sudo fdisk -l /dev/sda
Disk /dev/sda: 2.7 TiB, 3000592982016 bytes, 5860533168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x0003eff1

Device     Boot      Start        End    Sectors   Size Id Type
/dev/sda1             2048  209717247  209715200   100G  5 Extended
/dev/sda3        209922048 4075401215 3865479168   1.8T 83 Linux
/dev/sda4       4075401216 4532393983  456992768 217.9G 83 Linux
/dev/sda5             6144   89309183   89303040  42.6G 83 Linux
/dev/sda6         89311232   93530111    4218880     2G 82 Linux swap / Solaris
/dev/sda7  *      93532160  209717246  116185087  55.4G 83 Linux

Partition table entries are not in disk order.

I'm attaching a screenshot of the KDE partition manager because it gives a better view of the partition layout. (See #5 below.)

4) I'm not yet using Kubuntu, Fedora is installed and I'm not at all happy with it.

5) As to the partitions above, I'm planning during the impending install of Kubuntu on deleting sda5 (/tmp), sad6 (SWAP), and sda 7 (root partition), then replacing them with a new SWAP space double my memory size, and the rest of the freed space will be the root partition. I also have over 600G of space at the end of the drive where I want to add two additional partitions, /opt (~28.336G) and /var (all remaining space, ~604.6736G.) That will solve several space problems I've been having. I could also use some help getting the USB drive to boot up so I can install kbuntu, it's hanging on me during boot.

[oshunluvr]

This is because you're using MBR and it only supports 2.2TB or 2TiB. You have to switch to GPT formatting.

Questions:
1) Are you booting to this device?
2) If #1 is "yes" are you using GRUB or EFI to boot?
3) What does your current partition table look like? We need a detailed print out. The results of sudo fdisk -l /dev/sda should do it.
4) What version of Kubuntu are you using?

Answer these and we can help you convert.

[rwbehne1]

I'm trying to add additional partitions to my 3T drive, but I'm getting this error from the KDE partition manager:

Create a new partition (27.67 GiB, btrfs) on ‘/dev/sda’ 

Job: Create new partition on device ‘/dev/sda’ 

[FONT=Monospace]Failed to add partition ‘New Partition’ to device ‘/dev/sda’.[/FONT] 
[FONT=Monospace]starting sector number, 4532393984 exceeds the msdos-partition-table-imposed maximum of 4294967295[/FONT] 
[FONT=Monospace]Failed to add partition ‘New Partition’ to device ‘/dev/sda’.[/FONT] 
Create new partition on device ‘/dev/sda’: Error
Create a new partition (27.67 GiB, btrfs) on ‘/dev/sda’: Error

What does this mean, and what's the workaround?