Announcement

**oshunluvr** · Jan 29, 2015, 07:33 AM

My suggestion is to continue as you've planned (and nicely documented, I must say

). Once you have the data backed up, hold your breathe and run the conversion. I think is will happen almost immediately, or not more than just a minute or so. The conversion will automatically create a "snapshot" of the existing data along with the conversion.

Steps:

Unmount the target partition, run fsck on it, then run btrfs-convert:

Code:

sudo umount /dev/sdb1
sudo fsck.ext4 /dev/sdb1
sudo btrfs-convert /dev/sdb1

I would now run a quick test to be sure all is well:

Code:

sudo btrfs device scan
sudo btrfs filesystem show

The first command should return nothing and the second should return the device with it's size and UUID. Now edit fstab changing the mount to btrfs and I recommend these options:

rw,relatime,space_cache,compress

relatime isn't a btrfs thing, it just reduces time stamping. space_cache and compress will speed up access.

Once you have it mounted, do an ls and you should see all your files just as they were along with a snapshot labeled /ext4_saved . At this point as long as you've not changed any data on the drive, you can still re-mount it as ext4 or even rollback the conversion.

You can mount the snapshot separately by using the option subvol=ext4_saved. So an fstab entry to mount the subvolume to /mnt/data_snapshot that might look like:

Code:

UUID=99999999...    /mnt/data_snapshot btrfs rw,relatime,space_cache,compress,subvol=ext4_saved 0 0

If you are happy with the btrfs results, you can remove the snapshot:

Code:

sudo btrfs subvolume delete /mnt/data/ext4_saved

A bit about snapshots: Snapshot are not copies of anything initially, it just tells the filesystem to hold the meta data. As you delete data on the volume that you snapshot-ed, the space taken up by the snapshot will grow.

Example: Vinny has his data on /dev/sdb1 at /mnt/data. He makes a snapshot of it. Next, he deletes a movie file. A week later he changes his mind. He mounts the snapshot, copies the movie back to his data directory, and it's back. Or he could just mount the snapshot and play it there. Neat! One caution: snapshots are not really backups. Since they reside on the same filesystem as the original - anything that kills the original filesystem or drive kills the snapshot. Snapshots are great to prevent stupid mistakes (like deleting your favorite movie) or to rollback your install to it's previous state after installing something new.

Vinny, one more suggestion:

Once the conversion is complete and before you start using the new filesystem, I would move the files out of the root directory of the filesystem into a subvolume. With a drive that size, you can do a lot with it but keeping the data segregated will help keep your drive in order. I'm not sure if the automatically created snapshot is read-only or not. If it's not read-only (a simple touch command will reveal it with a read-only error) you can just mount the snapshot instead of the root volume and use it that way - deleting the files from the root volume. Here's why I would do this:

After the conversion, /mnt/data contains:

/Documents
/Pictures
/Videos
...
...
/ext4_saved

I would make a subvolume named @data and move all the files into it and delete the saved subvolume (or rename and mount the ext4_saved subvolume if it's rw and delete the files). Then a mount of /dev/sdb1 would look like

@data

Then go back to fstab and change the mount option for /mnt/data and add subvol=@data

Now you can add more subvolumes to /dev/sdb1 to do other things, like save backups or do a new install. An example: Grab a copy of 14.10 with plasma5. Boot to it and install it to /dev/sdb1 AND DO NOT FORMAT! It will allow you to select the /dev/sdb1 partition, using it as btrfs and set the mount point as "/" and make sure the format box is UNCHECKED! Then allow the install to complete. Once it's done, reboot to your normal install. Mount /dev/sdb1 (the whole partition, not a subvolume) and you will see this:

@
@data
@home

@ contains your new 14.10 with plasma5 install, @home contains the home folder for that install, and your data rests comfortably in @data. Very neat and organized.

To mount these new subvolumes, copy the same fstab line you created for @data and change the subvolume in the option to match the @ or @home. Easy.

I keep a subvolume on my big drive named @backups. Within it I have four folders: system, media, home, server. I then use rdiff-backup and a cronjob to make backups of my data at 2am on every Monday.

**vinnywright** · Jan 29, 2015, 06:16 PM

OK ,,,,,,so Geekiness cant wate

hears where we are .

data backed up to Ubuntu partition(/dev/sda1) took 10-15 min. to transfer the 88GB of data .

then.................

Code:

vinny@vinny-Bonobo-Extreme:~$ sudo umount /dev/sdb1

vinny@vinny-Bonobo-Extreme:~$ sudo fsck.ext4 /dev/sdb1
e2fsck 1.42.9 (4-Feb-2014)
Extra_Drive_1: clean, 4394/61054976 files, 26925854/244190208 blocks

vinny@vinny-Bonobo-Extreme:~$ sudo btrfs-convert /dev/sdb1
creating btrfs metadata.
creating ext2fs image file.
cleaning up system chunk.
conversion complete.

the "btrfs-convert" took 15 minutes , about the same as the data transfer .

now we have

Code:

vinny@vinny-Bonobo-Extreme:~$ sudo btrfs device scan
[sudo] password for vinny: 
Scanning for Btrfs filesystems

vinny@vinny-Bonobo-Extreme:~$ sudo btrfs filesystem show
Label: 'Extra Drive 1'  uuid: 030aba53-b498-4ff5-9e41-80699f3e2c02
        Total devices 1 FS bytes used 102.85GiB
        devid    1 size 931.51GiB used 931.51GiB path /dev/sdb1

Btrfs v3.12

so a simple

Code:

vinny@vinny-Bonobo-Extreme:~$ sudo mkdir /mnt/btrfs

vinny@vinny-Bonobo-Extreme:~$ sudo mount -t btrfs /dev/sdb1 /mnt/btrfs

and all files are hear

Code:

vinny@vinny-Bonobo-Extreme:/mnt/btrfs$ ls -l
total 4                                                                                                                                 
drwxr-xr-x 1 root  root   10 Jan 29 16:48 ext2_saved                                                                                    
drwxr-xr-x 1 vinny vinny 120 Oct 13 23:17 kubuntu [COLOR=#ff0000]<----------------- this was the origanal location of my ~/ directories and all data is intact.  [/COLOR]                                                                                     
drwx------ 1 root  root    0 Oct  2 16:55 lost+found

however I do not think their are any subvolume's yet ,,,,,,,,,,the ext2_saved contains a 931GB image ,,,,, whitch is strange as df shows 104GB used and 552GB free ?

Code:

vinny@vinny-Bonobo-Extreme:/mnt/btrfs/ext2_saved$ ls -l --block-size=M
total 105180M
-r-------- 1 root root 953868M Dec 31  1969 image


vinny@vinny-Bonobo-Extreme:/mnt/btrfs/ext2_saved$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3        52G   30G   20G  60% /
none            4.0K     0  4.0K   0% /sys/fs/cgroup
udev            7.8G   12K  7.8G   1% /dev
tmpfs           1.6G  1.3M  1.6G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.9G  2.4M  7.9G   1% /run/shm
none            100M   20K  100M   1% /run/user
/dev/sda1       296G   93G  189G  33% /media/vinny/Ubuntu
/dev/sdb1       932G  104G  552G  16% /mnt/btrfs

?snapshot maybe ,,,,, so I'm not sure about the "You can mount the snapshot separately by using the option subvol=ext4_saved. So an fstab entry to mount the subvolume to /mnt/data_snapshot that might look like:" unless your saying you can mount a snapshot as a subvolume ? yes/no ?

so for now I will just try to create a @data subvolume then wait for further direction from you @oshunluvr as to the correct way to move the data from /sdb1/kubuntu to @data/kubuntu and mount the @data in fstab ........... ?? or should that be just mount /devsdb1 and @data will show their ,,,,,,,I want access to the hole disk (O ya it's dynamically expandable so @data should suffice ,,, right )

time to quit staring at the screen ...............LOL

VINNY

OOPS you did explain that but witch way is best ,,,move /sdb1/kubuntu into @data and delete the ext2_saved and just mount /sdb1 with the subvol=@data ,,,,,, or what

**vinnywright** · Jan 29, 2015, 07:35 PM

OK after a rest and some food + a bit more thought ,,,,,,,,,,,,,,,,,,,,,,,,,,,,, I think I am over complicating this hole thing in my tiny pee brain .

if my thinking is on the right path I really do not need to do anything but add it(/sdb1) to my fstab , like so

Code:

/dev/sdb1    /mnt/btrfs    btrfs     rw,relatime,space_cache,compress     0    0

and I'm good ,,,,,,,,,,,,,,,,the ext2_saved snapshot can go (why do I nead a snapshot of data thats backed up to the /sda1)

and any installs I may want to do will install as @ and @home subvolumes anyway like this

Code:

vinny@vinny-Bonobo-Extreme:/mnt/btrfs$ ls -l
total 4
drwxr-xr-x 1 root  root    0 Jan 29 19:57 @data
drwxr-xr-x 1 root  root   10 Jan 29 16:48 ext2_saved
drwxr-xr-x 1 vinny vinny 120 Oct 13 23:17 kubuntu
drwx------ 1 root  root    0 Oct  2 16:55 lost+found

contained and out of the way

let me know if I'm missing something ,,,,,,,,,,but this seems ok right ,,,,,,, or will /dev/null-ing the snapshot be bad ,,,,as in thrashing things ?

VINNY

more reading ,,,O my head is swimming ,,,,,,,think I'll go with compress=lzo in the fstab the btrfs.Wiki is saying it's faster ,,,,but a bit lower compression .

**oshunluvr** · Jan 29, 2015, 09:39 PM

Do this:

sudo btrfs subvolume list /mnt/btrfs

This will list all the subolumes.

If, in fact as I think, /mnt/btrfs/ext2_saved lists as a subvolume, you can remove it instantly with:

sudo btrfs subvolume delete /mnt/btrfs/ext2_saved

Leaving it will not thrash anything or cause any slow down of your system. What happens is; as the source volume the snapshot was taken of has files deleted or changed, they will not actually be deleted and changed files will have both a new and old copy. They will "move" into the snapshot folder (the actual data doesn't move and remains intact, the file system just lists the file in a different place). Eventually, the snapshot will "grow" to be the size of the original volume - assuming you eventually delete and/or edit every file in it. Also you can delete the lost and found folder - it's left over from the extN filesystem.

And a snapshot is a subvolume. Difference is It contains files from a pre-set location (what ever you snapshot-ed) rather than starting empty like a newly created subvolume is.

Let say you're about to install a new program to your Kubuntu that lives on a btrfs filesystem. This new program is one you're not sure about, so you make a snapshot of the root folder then do the install. Well you were right and the new program mucks up a bunch of stuff. You simply re-name the root folder, give the snapshot the name your root folder used to have, re-mount everything and you're back to the beginning.

One of the great but potentially confusing things about subvolumes is they can be mounted, thus protecting the root folder and other subvolumes from access.

Example;

Root filesystem is /dev/sdb1 and is mounted at /mnt/btrfs. This root filesystem contains four subvolumes: @, @home, ext2_saved, and kubuntu

Anyone with access to /mnt/btrfs can see all the files on all the 4 subvolumes. But lets say you don't want that. ext_2saved should be protected from everyone and the files in kubuntu belong only to vinny.

So instead of mounting /dev/sdb1 to /mnt/btrfs, you only mount /dev/sdb1 subvol=kubuntu to /home/vinny/data. Now, unless another user has sudo rights, they can't get to ext2_saved at all and Vinny's file are mounted in his home away from meddling fingers. Since you boot to /dev/sdb1 subvol=@ and /dev/sdb1 subvol=@home contains the home folders, they're mounted as / and /home just like you would any partition.

You want file security from your other users? Just make everyone's home an individual subvolume and mount it when they log in. Then as far as anyone can see, no one else has files on the system at all. Neato!

**vinnywright** · Jan 29, 2015, 09:57 PM

Code:

vinny@vinny-Bonobo-Extreme:~$ sudo btrfs subvolume list /mnt/btrfs
[sudo] password for vinny: 
ID 256 gen 23 top level 5 path ext2_saved
ID 693 gen 25 top level 5 path @data

if it "grows" as I assess/delete files in /sdb1/kubuntu it will eat up space will it not .......O wate I read that rong the files dont get moved or copyed just the info changes places ?......I can remove it safely or not ?

VINNY

WOW you added a lot to that post wile I was posting ..........trickery

**vinnywright** · Jan 29, 2015, 10:04 PM

OK read it all

this time ,,,,,,,,,,,,,yes that is cool .

what do you think of the fstab compress=lzo ?

VINNY

**oshunluvr** · Jan 30, 2015, 08:51 AM

I had not used lzo in the past because there was some problem with grub and an lzo compressed kernel it at boot time - years ago. I never bothered to change back to it. Everything seems fine now. "Compress" (zlib compression) does best compression at a slower speed, "Compress=lzo" is faster but not a good at compression. Both cause the filesystem to run faster except when writing when there's a very small hit.

I doubt I'll see a difference, but I'm changing to lzo now.

**oshunluvr** · Jan 30, 2015, 08:57 AM

BTW, if you change from non-compressed to compressed or change compression type. I think you have to run defrag for it to take effect on older files. Otherwise the change only happens to new files.

Also, regular "df" may not correctly report your volume sizes. Use "btrfs flesystem df /mnt/sdb1" to get a truer size.

**oshunluvr** · Jan 30, 2015, 09:25 AM

Reading some later notes: I found that adding

autodefrag,

to mount options is good too. Here's a list of options

https://btrfs.wiki.kernel.org/index.php/Mount_options

**vinnywright** · Jan 30, 2015, 02:05 PM

humm could have sworen I read somewhere defrag was not a good idea yet.

more reading to do ........

but not tonight , we have somewhere to go

VINNY

**oshunluvr** · Jan 30, 2015, 02:20 PM

I read one post where the guy claimed after defragging, it slowed down. I don't see any difference here.

I suspect it's not really needed is most cases. You definitely don't want to do it on a drive containing a vm disk or a very large database. The filefrag utility does not work on btrfs so there's no real way to tell if your 'fragged or not.

**oshunluvr** · Jan 30, 2015, 02:21 PM

I looks like running the "balance" command might do the trick also - defrag a bit plus add or change the compression.

**vinnywright** · Jan 30, 2015, 04:08 PM

I new I read this somewhere ,,,,,,,it's not good for snapshots as well

# btrfs filesystem defragment [-c] file_name ... Note
You can set up automatic defragmentation by specifying the autodefrag option when you mount the file system. However, automatic defragmentation is not recommended for large databases or for images of virtual machines.
Defragmenting a file or a subvolume that has a copy-on-write copy results breaks the link between the file and its copy. For example, if you defragment a subvolume that has a snapshot, the disk usage by the subvolume and its snapshot will increase because the snapshot is no longer a copy-on-write image of the subvolume.

VINNY

**vinnywright** · Feb 09, 2015, 06:52 PM

at the beginning of this test/journey I sead it should round off with getting all the grub installs chain loading together ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,and we finally got their

I can go one to the other (3) all day without ever booting ,,,,,,,,,,,,,,,LOL

VINNY

Announcement

convert ext4 with data to btrfs

convert ext4 with data to btrfs

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment