USB drive nightmare

[GreyGeek]

It seems that with kparted one cannot format the drive itself without creating a partition. So, instead of formatting /dev/sdd with Btrfs one must create a partition and format /dev/sdd1. BUT, using mkfs.btrfs one can format sdd without creating sdd1.
For grins and giggles I formatted a 64Gb Sandisk.

[FONT=monospace][COLOR=#000000]$ sudo mkfs.btrfs -f -d single /dev/sdd[/COLOR]
btrfs-progs v4.9.1
See http://btrfs.wiki.kernel.org for more information.

Label:              (null)
UUID:               3b0030b7-550b-4894-aada-be3e0343df53
Node size:          16384
Sector size:        4096
Filesystem size:    59.16GiB
Block group profiles:
  Data:             single            8.00MiB
  Metadata:         DUP               1.00GiB
  System:           DUP               8.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata
Number of devices:  1
Devices:
   ID        SIZE  PATH
    1    59.16GiB  /dev/sdd
[/FONT]

But notice that the usable size remains the same as when I created an sdd1 partition.

[dibl]

AFAIK, this old How-To is still all valid. Omit the steps to make it bootable, if that is not needed, and omit the steps to install an OS on it, if that is not needed. But using dd to zero out the mbr area will solve the "Sandisk crud" issue, if that is applicable, and the rest of the procedure using gparted will result in a single-partition formatted USB stick (or hdd). As mentioned above by others, "Linux filesystem" is a non-descriptive term, as there are at least a dozen of them. The media in a USB stick are not designed for more than a few thousand rewrites, so a journalling filesystem is not advisable unless you disable the journal. FAT32 is a perfectly functional choice for this scenario, as is ext2 if you'll never need to use it on a Windows system.

[GreyGeek]

On the new 64Gb Sandisk I used I also tried dd to zero the whole stick and it still resulted in only 59.15Gb usable. Isn't MBR only 540 bytes? (Early riser?)


Sent from my iPhone using Tapatalk

[dibl]

Hi GG. Wikipedia says "512 or more bytes". Reviewing that article, it appears that the original mbr has grown over time, so maybe it was 540 bytes at a point in time.

I wonder if the difference in reported size that you are seeing is an artifact of the difference between the manufacturer's advertised capacity, based on (1000x1000)=1,000,000 bytes per MB, and the OS-measured capacity, based on (1024x1024)=1,048,576 bytes per MiB. Try dividing 64,000,000 by 1,048,576. ;-)

[GreyGeek]

I considered that too, but the diff between 64Gb (64,000,000,000) and 59.15Gb is more than can be accounted for by a multiple of 1,024.

[dibl]

Another notion -- if that USB stick is formatted with a journaling filesystem, maybe the OS is deducting the space for the journal when reporting available capacity.

[GreyGeek]

I have tried ext4, msdos and btrfs. They all give a max space of 59.15 of 59.16Gb.

[dibl]

Try

fdisk -lu

in a root terminal, or with sudo prefix. It should tell exactly the total bytes, as in

Disk /dev/sde: 1.9 GiB, 2021654016 bytes, 3948543 sectors

[vinnywright]

OK ,,,,we all know (the old hats),,,or should that you NEVER get all the advertised space of a drive , their is all ways overhead of some form or a another and the larger the drive so the larger the overhead .

BUT hear is an example of one case of extra overhead getting reclaimed ...

this is a (advertised as) 16GB usb stick with a live-system on it made with mkusb ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,as seen by GParted



I used GParted to "delete" all the partitions ...



I then tryed to add new partitions of 3 types , ext4 , fat32 and NTFS ,,,,,,all left a 1MB unallocated space .







I then used dd to zero the drive with

sudo dd if=/dev/zero of=/dev/sdc

this took a long time ,,,,,,,,,,BUT on the next attempt to give it a fat32 formated partition we get NO unallocated space.



and this is how "parted" sees it ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,



now if you look in the parted output just above the "SMI USB DISK" you will see how "parted" sees my 1TB storage drive ,,,,,,,,, it's reporting 1000GB ,,,, BUT this is how GParted sees it .



as we see GParted is reporting 931.51GB size ,,,,,,so as I sead the larger the drive , so the larger the overhead ,,,,,,,,,,,,,,,,suck it up , expect it ,,,,and get on with your bad self and do what you do

VINNY

[GreyGeek]

Vinny is taking us to the woodshed!

[vinnywright]

vinny is taking us to the woodshed! :d

lol

VINNY

[Qqmike]

For this sort of thing, with a flash drive,

I would always, first, see if the manufacturer has a removal tool, like the U3 removal tool with SanDisk.
Then/or
Use GParted to delete partitions and clear things out.
Then/or
Use dd ...
... to zero things out
... to see what is where
... to see what's at the end of the drive (like what's hidden).
dd, as in my how-to
The dd Command
https://www.kubuntuforums.net/showthread.php?t=27726

Part 2 Examples (Part 2 is in Reply #1)
See Part 2, Reply #1, for an extensive list of examples, including:
Zero-out the hard drive or flash drive, Zero-out the free space on a partition, Clean up your home, Clean a Partition Before Reusing It or Deleting It, Clean the Master Boot Record, Wipe a file before deleting it (to prevent recovery), Cloning a [bootable] flash drive or a hard drive, Master boot record: viewing and backup/restore, Print a partition boot sector, Creating files, Check a file for errors, Print bytes of a file to the screen, Look for non-zero entries on a drive (HDD or flash drive), Search for a string in your /home partition, Check a random part of a drive to see what's there, Write numbers to the end of a disk, md5sums and dd, Using the Bash command sync after a dd copy, What's at the end of YOUR hard drive?, Backup your Kubuntu OS to your separate /home partition, Bad disk sectors

[Qqmike]

(Deleted because it already posted above. Trouble with this forum posting ...)

[SumBloke]

Some great information in this thread!

Im still having serious issues. Mainly with one Kubuntu pc reading/writing to either ext4 or Fat32 USB drives. Its not the USB ports as all of them mangle both drives. Its not the physical USB drives as both of them get mangles on this pc only.

Is there a setting on the PC that could be causing this to do with 'mounting at boot' or perhaps in the way Kubuntu is configured (mistakenly) to access the drives that is wrong? Anything I could look into you can think of? I don't have much more info on what is happening to the USB sicks other than I partition and cleanly format them, in either format, try to copy stuff over from other machines which works fine, just when I go through this particular PC it will sometimes allow me to read or write once, then everything stops. I did see a ton of ghost files with chines character file names once on this particular Linux PC.

All PCs espceially the Windows ones are kept as secure and up to date as possible. And, I'm no longer concerned about that hidden used space!

[GreyGeek]

What do you mean when you write "mangled"?

chines character...

Are your USB sticks Chinese knockoffs that advertise more capacity than they actually have? Like advertising 4Gb when they actually are only 250Mb, or less ...
https://www.reddit.com/r/mildlyinter...tual_capacity/

/dev/ and its subdirectories are owned by root, so you can only use "mount /dev/sdd1 /mnt", for example, by prefixing "sudo " in front of it. Having mounted it you can, as root, use mc (Midnight Commander) or other file manager tools that are allowed to be run as root, Dolphin not being one of them, sadly:

$ kdesudo dolphin
Executing Dolphin as root is not possible.
$

"kdesudo krusader" does open up Krusader with root privildeges, so a useful GUI filemanager is still available if you choose not to use mc or the CLI. Krusader looks like a GUI to mc.

Even if you attempt to mount a USB device to a subdirectory owned by you under your home account, say ~/tmp, tmp's ownership and group will be temporarily be converted to root by the "sudo mount /dev/sdd1 ~/tmp" command. However, any files you copy to ~/tmp will retain their ownership and permissions if you allow it. When you drag and drop from your home account to a mounted USB stick your files will retain their ownership, group and permissions, unless you change the default settings.

PS. - You do not want to copy your home account to ~/tmp because tmp is part of your home account. You'll be like the snake eating its own tail.