Disk benchmarks

[Don B. Cilly]

Just idle curiosity basically.
Checking out my disks' performance:

not@all:~$ sudo hdparm -t /dev/sda
[sudo] password for not: 

/dev/sda:
Timing buffered disk reads: 432 MB in  3.00 seconds = 143.79 MB/sec
not@all:~$ sudo hdparm -t /dev/sdb

/dev/sdb:
Timing buffered disk reads: 314 MB in  3.00 seconds = 104.50 MB/sec
not@all:~$ sudo hdparm -t /dev/sdc

/dev/sdc:
Timing buffered disk reads: 1050 MB in  3.00 seconds = 349.61 MB/sec

Sda is a 1TB WD disk, about 5 years old. It has failed me a few times. I use it for arcane distros and storage. Very seldom.
Sdb is another WD, 320G, about 8 years old. I don't use it often, but sometimes I do, it hasn't failed yet.
Sdc is a Crucial SSD, 500G, about six months old.

My mobo has four SATA slots. Two SATA2 and two SATA3. Sdb and sdc are on the SATA3 ones. At least, I think...
I was expecting better performance from the SSD... but it is fast.

Are these any good? What do your hdparm(s) say?

Have I got the SATA ports mixed up? The (supposedly) SATA3 ones are grey, the other two, black.
I remember looking it up when I installed the stuff, but... it wasn't all that clear :~$

[oshunluvr]

My desktop;

stuart@office:~$ for dr in `lsblk -d |grep -v -- 'loop\|sr\|NAME'`; do sudo hdparm -Tt '/dev/'$dr; done
[sudo] password for stuart: 

/dev/nvme0n1:
 Timing buffered disk reads: 4686 MB in  3.00 seconds = 1561.78 MB/sec

/dev/sda:
 Timing buffered disk reads: 430 MB in  3.01 seconds = 143.01 MB/sec

/dev/sdb:
 Timing buffered disk reads: 416 MB in  3.01 seconds = 138.40 MB/sec

/dev/sdc:
 Timing buffered disk reads: 1608 MB in  3.00 seconds = 535.69 MB/sec

/dev/sdd:
 Timing buffered disk reads: 1610 MB in  3.00 seconds = 536.09 MB/sec
stuart@office:~$

Drive info:

nvme=512gb Samsung SSD 970 PRO 512GB, Power_On_Hours - 3780
sda=2tb WDC WD2002FAEX-007BA0, Power_On_Hours - 62116
sdb=2tb WDC WD20EFRX-68AX9N0, Power_On_Hours - 49543
sdc=256gb Samsung SSD 840 PRO Series, Power_On_Hours - 46471
sdd=256gb Samsung SSD 840 PRO Series, Power_On_Hours - 39125

The Samsung SSDs were the fastest available at the time of purchase. I guess that shows.

[oshunluvr]

On my server;

sda = 165.03 MB/sec, 6tb Model=WDC WD60EFRX-68MYMN1, Power_On_Hours -       31069

sdb = 174.61 MB/sec, 6tb Model=WDC WD60EFRX-68L0BN1, Power_On_Hours -       17329

sdc = 237.98 MB/sec, 10tb Model=WDC WD101KFBX-68R56N0, Power_On_Hours -       17053

sdd = 331.96 MB/sec, 60gb Model=Patriot Torch, Power_On_Hours - 11342

This ssd, sdd, is not much faster than the hdds. However I only paid $30 for it so you get what you pay for.

I haven't checked this stuff in quite awhile. I don't remember exactly when I bought the second 6tb drive but hard to believe I bought the 10tb drive so close after the second 6tb, but I guess I did based on the hours.

This server is much lower powered (CPU, RAM, etc.) than my desktop but doesn't seem to make much of a difference for the drives. I access it only through the network so drive specs are less important. The 2 2tb drives on my desktop were pulled from the server when the second 6tb drive was installed.

The server case has 4 3.5" hot swap bays and one internal 2.5 inch drive mount. Drive history of the server (first and second gen):
Initial 2010 build: 4x500GB WD Blue (3 yrs old, pulled from desktop)
2011: +2x1TB WD Black, -4X500GB Blue (Blues started failing, Blacks pulled from desktop)
2012: +1x2TB WD Red
2014: +1x2TB Black (pulled from desktop) -2x1TB Black (60,000+ hours)
2015: +1x6TB WD Red
2016: +1x60GB Patriot Torch (gen 2 build, new mobo, CPU and RAM)
2017: +1x6TB WD Red
2018: +1x10TB WD Red -2x2TB WD Red/Black (60,000+ hours moved to desktop for backup and playground)

The SSD is the boot device, all the other drives are redundant storage. Prior to the SSD two of the HDDs were bootable. Today, the 6TB drives are bootable but the SSD is the primary boot device.

Total data/backup storage:
2010: 1/1 TB
2011: 1/1 TB
2012: 2/2 TB
2014: 2/2 TB
2015: 6/4 TB
2017: 8/8 TB
2018: 10/12 TB
TBA: 16/16 TB

My methodology is to move drives from service to backup as they age while increasing capacity at the same time by increasing drive size. For whatever reason, large NAS hard drives have gotten expensive. I've paid $250 per drive for all of them +/- $50 except for the 10TB that was nearly $400. It's also an enterprise class drive where the previous Red drives were consumer grade.

I'm comfortably holding at about 7TB of used space so I have no need to expand anytime soon. IME, those Red 6TB drives will last for several more years. The Blue drives started failing at 3.5 years and all failed but one within a couple months. The Black drives are high performance but have lasted a long time, one 2TB still running without a hiccup. The 2 1TB Black drives I discarded because I no longer had a place for them. The Red drives are a fairly new WD product and are meant for NAS use and they've been super reliable. Haven't had one die yet.

[claydoh]

My spinner drives seem awfully fast here.

/dev/sda:
 Timing cached reads:   33434 MB in  1.99 seconds = 16836.71 MB/sec
 Timing buffered disk reads: 598 MB in  3.00 seconds = 199.11 MB/sec

[FONT=monospace][COLOR=#000000]dev/sdb:[/COLOR]
 Timing cached reads:   33958 MB in  1.99 seconds = 17102.25 MB/sec
 Timing buffered disk reads: 1224 MB in  3.00 seconds = 407.48 MB/sec

[/FONT][FONT=monospace][COLOR=#000000]/dev/sdc:[/COLOR]
 Timing cached reads:   33798 MB in  1.99 seconds = 17019.68 MB/sec
 Timing buffered disk reads: 368 MB in  3.00 seconds = 122.57 MB/sec

[/FONT][FONT=monospace][COLOR=#000000]/dev/nvme0n1:[/COLOR]
 Timing cached reads:   34646 MB in  1.99 seconds = 17451.10 MB/sec
 Timing buffered disk reads: 2866 MB in  3.00 seconds = 954.90 MB/sec[/FONT]

[FONT=monospace]System: refurbished [/FONT]HP Pavilion 590-p0097c[FONT=monospace]
[/FONT]sda        TOSHIBA DT01ACA2            2Tb      OEM drive
sdb        PNY ELITE PSSD                 447gb   portable SSD usb 
sdc        WD My Book 1140                 1Tb       old portable backup drive

nvme0   WD Black WDS250G2X0C              256Gb m.2 nvme, reports as a Sandisk

[Don B. Cilly]

Why are your cached reads 10 times as fast as mine?

dev/sda:
Timing cached reads:   3274 MB in  1.99 seconds = 1642.87 MB/sec
Timing buffered disk reads: 516 MB in  3.01 seconds = 171.45 MB/sec

/dev/sdb:
Timing cached reads:   3176 MB in  1.99 seconds = 1593.05 MB/sec
Timing buffered disk reads: 316 MB in  3.01 seconds = 104.94 MB/sec

/dev/sdc:
Timing cached reads:   3194 MB in  1.99 seconds = 1601.50 MB/sec
Timing buffered disk reads: 1062 MB in  3.00 seconds = 353.57 MB/sec

And... the Power_On_Hours info, is it from smartctl or can it be obtained with hdparm?

[oshunluvr]

Power on hours is from smartctl:

sudo smartctl --all /dev/FOO | grep Power_On_Hours

Re cached reads, from the manual:

This displays the speed of reading directly from the Linux buffer cache without disk access. This measurement is essentially an indication of the throughput of the processor, cache, and memory of the system under test. If the -t flag is also specified, then a correction factor based on the outcome of -T will be incorporated into the result reported for the -t operation.

I didn't bother getting mine as they're all roughly the same:

My desktop:
Timing cached reads: 38240 MB in 1.99 seconds = 19259.78 MB/sec

The answer is his computer is faster than yours and mine faster than his. My server is quite a bit slower:

Timing cached reads: 15370 MB in 2.00 seconds = 7701.11 MB/sec

Desktop: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz, 16GB RAM
Server: Intel(R) Core(TM) i3-4160T CPU @ 3.10GHz, 8GB RAM

I'm guessing Clays CPU is a bit slower than mine and yours more so.

[Don B. Cilly]

The answer is his computer is faster than yours and mine faster than his.

Very clear, thanks.

I also got smartctl to tell me what my SATA ports are:

/dev/sda SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s) 143.79 MB/sec
/dev/sdb SATA Version is:  SATA 2.5, 3.0 Gb/s (current: unspecified) 104.50 MB/sec
/dev/sdc SATA Version is:  SATA >3.2, 6.0 Gb/s (current: 6.0 Gb/s) 349.61 MB/sec

/dev/sda Power_On_Hours - 15059
/dev/sdb Power_On_Hours - 40061
/dev/sdc Power_On_Hours - 2582

So I was wrong, sda (the newer of the two HDDs) is on the SATA 3 one. Which is irrelevant, I don't use the HDDs much. And anyway, even the SSD, it could be connected to a SATA 2 one - I mean, I think 3 Gbps = 375 MB/s, my SSD only does 349...

[GreyGeek]

/dev/sda:
Timing buffered disk reads: 1610 MB in 3.00 seconds = 536.59 MB/sec (Samsung 860 EVO 500GB SSD)

/dev/sdb:
Timing buffered disk reads: 266 MB in 3.01 seconds = 88.35 MB/sec (Toshiba 750Gb HD)

/dev/sdc:
Timing buffered disk reads: 820 MB in 3.01 seconds = 272.68 MB/sec (Samsung EVO 860 500Gb SSD why should it be only 1/2 as fast as sda, the exact same drive? I suspect that it is because that drive is in the HDCADDY which replaced the CDROM.)

~$ sudo fdisk -l
Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 860 (Power on time 283 days 22 hours TBW=10.63 TB)
...Disklabel type: dos
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 976771071 976769024 465.8G 83 Linux


Disk /dev/sdc: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 860 (Power on time 90 days 6 hrs TBW=435.71GB)
...Disklabel type: dos
Device Boot Start End Sectors Size Id Type
/dev/sdc1 2048 976766975 976764928 465.8G 83 Linux


Disk /dev/sdb: 698.65 GiB, 750156374016 bytes, 1465149168 sectors
Disk model: TOSHIBA MK7559GS
...Disklabel type: gpt
Device Start End Sectors Size Type
/dev/sdb1 2048 1465143295 1465141248 698.6G Linux filesystem

[oshunluvr]

GG, what's the out put of:

for dr in `lsblk -d |grep -v -- 'loop\|sr\|NAME'`; do sudo smartctl -a '/dev/'$dr | grep 'Device Model\|SATA' ; done

EDIT: Included model in output

[GreyGeek]

SATA Version is: SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
SATA Version is: SATA 2.6, 3.0 Gb/s (current: 3.0 Gb/s)
SATA Version is: SATA 3.2, 6.0 Gb/s (current: 3.0 Gb/s)

sudo lshw:

*-sata
description: SATA controller
product: 7 Series Chipset Family 6-port SATA Controller [AHCI mode]
vendor: Intel Corporation
physical id: 1f.2
bus info: pci@0000:00:1f.2
logical name: scsi0
logical name: scsi2
logical name: scsi3
version: 04
width: 32 bits
clock: 66MHz
capabilities: sata msi pm ahci_1.0 bus_master cap_list emulated
configuration: driver=ahci latency=0
resources: irq:30 ioport:5088(size=8) ioport:5094(size=4) ioport:5080(size=8) ioport:5090(size=4) ioport:5060(size=32) memory:d3617000-d36177ff


*-disk:0
description: ATA Disk
product: Samsung SSD 860
physical id: 0
bus info: scsi@0:0.0.0
logical name: /dev/sda
version: 1B6Q
serial: S3Z1NB0K453930Z
size: 465GiB (500GB)
capabilities: partitioned partitioned:dos
configuration: ansiversion=5 logicalsectorsize=512 sectorsize=512 signature=e00dfb49
*-volume
description: Linux filesystem partition
physical id: 1
bus info: scsi@0:0.0.0,1
logical name: /dev/sda1
logical name: /
logical name: /home
capacity: 465GiB
capabilities: primary bootable
configuration: mount.fstype=btrfs mount.options=rw,relatime,ssd,space_cache,subvolid =486,subvol=/@home state=mounted

*-disk:1
description: ATA Disk
product: TOSHIBA MK7559GS
vendor: Toshiba
physical id: 1
bus info: scsi@2:0.0.0
logical name: /dev/sdb
version: 3J
serial: 32RHP0TGT
size: 698GiB (750GB)
capabilities: gpt-1.00 partitioned partitioned:gpt
configuration: ansiversion=5 guid=cc604866-1b1d-4b67-b099-576fde2a69ce logicalsectorsize=512 sectorsize=4096
*-volume
description: EFI partition
physical id: 1
bus info: scsi@2:0.0.0,1
logical name: /dev/sdb1
serial: 5fa5762c-9d66-4fdf-ba8f-5c699763e636
capacity: 698GiB

*-disk:2
description: ATA Disk
product: Samsung SSD 860
physical id: 0.0.0
bus info: scsi@3:0.0.0
logical name: /dev/sdc
version: 3B6Q
serial: S3Z1NY0M803506Z
size: 465GiB (500GB)
capabilities: partitioned partitioned:dos
configuration: ansiversion=5 logicalsectorsize=512 sectorsize=512 signature=dc864468
*-volume
description: Linux filesystem partition
physical id: 1
bus info: scsi@3:0.0.0,1
logical name: /dev/sdc1
capacity: 465GiB
capabilities: primary

[oshunluvr]

Well there's your answer. The second SSD is hooked to a slower port. Bummer, but it's still faster than a spinner.

[GreyGeek]

Well there's your answer. The second SSD is hooked to a slower port. Bummer, but it's still faster than a spinner.

At least 3X !!

And, I don't mount either my 2nd or 3rd drive in fstab so the spinner rarely spins up unless I am doing a btrfs send & receive to it, which is its only purpose.

[oshunluvr]

Looking closer, it may not be the port but rather the tray as you suspected. Hard to tell what is what in that obscure data. Either way, like you said - at least it's 3x a platter

[Schwarzer Kater]

kater@PC-Kubuntu2004:~$ sudo hdparm -t /dev/sda
/dev/sda:
Timing buffered disk reads: 784 MB in  3.00 seconds = 261.11 MB/sec

512GB Kingston KC600 SSD (made 2019) @ SATA2 (computer made 2009)

[Schwarzer Kater]

… and my friend's computer:

karin@TC-M720s:~$ sudo hdparm -t /dev/sda
/dev/sda:
Timing buffered disk reads: 1580 MB in  3.00 seconds = 526.20 MB/sec

512 GB Samsung 860 Pro SSD @ SATA3

karin@TC-M720s:~$ sudo hdparm -t /dev/nvme0n1
/dev/nvme0n1:
Timing buffered disk reads: 6858 MB in  3.00 seconds = 2285.15 MB/sec

256 GB WDC PC SN730 @ M.2 / NVMe

… and I also got the following error message concerning the M.2 SSD:
"HDIO_DRIVE_CMD(identify) failed: Inappropriate ioctl for device"

Any quick thoughts?