| |
π
2012-08-15 01:01
in Linux
For indexing purposes, I didn't change the title, but I don't recommend the OCZ Vertex 4 anymore, it does lock up under certain workloads. My new drive is the Samsung 840 Evo which performs fine where the 830 absolutely did not for me.
Please also read this very good report from Luke on SSD unreliabilities.
Crucial C300 sudden death and full data loss
So, before people tell me "you should have bought intel", I got the C300 from work and installed it because I trusted it had been pre-qualified and was good. Turns out, I was one of the people doing the pre-qualifying :) Later, I did look at intel for my drives, but they were lacking in size capacity when I was looking for 512GB and later for 1TB. Too bad because they've been shown to be the most or amongst the most reliable drives so far.
I wasn't using TRIM with the C300 at the time, because it's not necessarily recommended if you use dm-crypt ( http://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html ). The C300 likely did not appreciate that I filled it a few times and that it had to use its garbage collection algorithm to reclaim blocks (when it was getting full, it was very very slow, as in 100x slower than a spinning drive at times).
In the end, it died after fewer than 3 months of use, where it just disappeared from the SATA bus. Some magic 'plug the power but don't do data' for 20mn or so, several times, brought it back (after several days of trying) long enough for me to see it could still talk SATA and that all my partitions and data were gone. Even then, I wasn't able to really usefully talk to it anyway.
If you read up on this, and hear Crucial support recommending that you let the drive sit at the bios for 20mn once it a while so that its garbage collection can kick in after 15mn of it not being used, you'll understand that you should run away in the other direction rather than buying one of those drives: http://forums.crucial.com/t5/Solid-State-Drives-SSD/Bios-is-not-recognize-crucail-m4-SSD-64GB/td-p/63835
Maybe the C400 sucks less, but once burnt, twice shy, and the C300 was so bad, that I just don't trust Crucial to write proper firmware at this point.
Samsung 830 512GB severe random IO slowness with linux and windows
So, I then did some reading about SSDs and shopping around. I found that the Intel did not come in 512GB (it was a bit smaller), and that while Intel ruled SSDs early on, it hasn't kept up in performance and offerings. Also, they use older sanforce controllers that do not do well with software encrypted data (exactly what I had).
From online reading, the next best player was Samsung, so I got an 830 512GB drive from them, and was quite hopeful after reading reviews and how they designed and manufacture all the parts themselves.
That decision came back to bite me. I spent too many hours (between 10 and 20) with samsung tech support, doing many benchmarks, and working with the linux community, because its random IO performance was bad. How bad? Well, how does 5 times slower than my hard drive in the same laptop sound?
On time spent with the linux community debugging this with multiple filesystems, laptops, and tweaks, I'm not kidding. See a small sample:
http://www.spinics.net/lists/dm-crypt/msg04403.html
http://www.spinics.net/lists/dm-crypt/msg04406.html
http://www.spinics.net/lists/dm-crypt/msg04408.html
http://www.spinics.net/lists/dm-crypt/msg04436.html
http://www.spinics.net/lists/linux-btrfs/msg18009.html
http://www.spinics.net/lists/linux-btrfs/msg18168.html
http://www.spinics.net/lists/linux-btrfs/msg18204.html
http://www.spinics.net/lists/linux-btrfs/msg18238.html
http://www.spinics.net/lists/linux-btrfs/msg18240.html
http://permalink.gmane.org/gmane.comp.file-systems.ext4/33491
http://permalink.gmane.org/gmane.comp.file-systems.ext4/33637
There is one problem that might be linux specific: the drive is very slow for encrypted block device access without:
blockdev --settra 8192 /dev/mapper/cryptroot
http://www.spinics.net/lists/dm-crypt/msg04408.html
http://www.spinics.net/lists/dm-crypt/msg04436.html
But past that, I found no other problems, and replicated the problem with ntfs, and even after putting the SSD in a windows desktop with Windows 7 64bit. Samsung's old Benchmark registered so low on their own Random IO test that the bar graph showing speed was a single line at the '0' mark.
Samsung support had no idea why I saw this. They suggested I get another drive in case mine was bad.
The new drive had identical problems and in the end I had to return both. Yes, the drives had the latest firmware. The test below shows lower block IO because it was on SATA2, not SATA3 like in my laptop but I was only worried about Random IO.
 
I have no idea why those drives were so unexplainably slow on random IO reads (not even writes) when block IO was fast (500MB/s). I don't know how they work for others, but they failed to perform for me on 3 machines, 4 filesystems, and 3 operating systems.
That was a lot of time wasted, and I'm not impressed with Samsung anymore :(
Samsung 840 Evo 1TB (bought in November 2013)
As explained below, the OCZ drive was fast and didn't have the random IO performance issue that the Samsung 830s had for me, but the Vertex ended up locking up every few days after 3-6 months of use, needing replacement. When I got tired of that, I eventually upgraded to a 1TB Samsung 840 Evo.
Non surprisingly, it's the fasted drive of them all, but it's over a year newer. With an upgrade to linux 3.11 in the meantime, encrypted btrfs on that drive is almost as fast as native raw speeds.
OCZ 512GB Vertex 4 vs Samsung 830 512GB vs Samsung 840 EVO 1TB linux tests with btrfs, ext4, and dmcrypt
Update: the drive is fast, it never really lost data (except apparently the last few blocks written before locking up), but I had 3 drives eventually go into a mode where they lock up and my laptop needs to be power cycled under certain workloads (mencoder is one for me). They always recovered, only lost the last bits of data being written when the drive locked up, but that's just no good. I replaced the drive 3 times and eventually switched to a 1TB Samsung 840 Evo.
Old text from July 2012:
After more research still, this drive came highly recommended too. It's the fastest from OCZ and the most expensive (they have slower cheaper ones). And yet, the most expensive one was $100 cheaper than the Samsung, and 36 times faster for random IO reads. Oh, also the OCZ has 5 year warranty instead of 2 for Samsung.
There are only crude tests, but they are the ones that mattered to me to validate that the OCZ didn't have severe problems in my hardware like the Samsung 830 did.
Here are optimizations I used:
fdisk cylinders were chosen so that they were multiple of 512 to make sure I was aligned with internal block write sizes
btrfs was created with mkfs.btrfs and mounted with mount -o compress=lzo,discard,nossd,space_cache,noatime
ext4 was created with mkfs.ext4 -O extent -b 4096 -E stride=128,stripe-width=128 and mounted with mount -o discard,noatime
cryptsetup luksFormat --align-payload=8192 -c aes-xts-plain /dev/sda2
cryptsetup luksOpen --allow-discards /dev/sda4 cryptroot
The OCZ was upgraded to Firmware Version 1.5, and the Samsung 830 claimed to have the latest firmware as of 2012/08 (that was version CXM03B1Q as reported by smartmontools). The Hitachi Hard drive in my laptop is a slow 5400RPM model.
Long story short, the OCZ is super fast, worked great in my tests, and was cheaper than the Samsung and the Crucial. Win all around.
Scanning 15K inodes with the Vertex 4 (vs Samsung and spinning hard drive)
Summary of results:
Samsung 830 is so slow, it doesn't make sense
OCZ Vertex 4 is super fast
Samsung 840 Evo 1TB more than a year later is much much faster and easily beats the Vertex 4
for reasons that I ignore, encrypted btrfs on my HD was 5x slower than unencrypted in the du -s case.
dmcrypt on linux makes btrfs random IO 50% slower
random seeks/IO on ext4 in this du test is 2x faster than btrfs on SSD, and 3x faster on HD.
At the same time, I'm pretty sure btrfs got faster between kernels 3.5 and 3.11 (this is supported by btrfs now being faster than ext4 when it was slower before
OCZ Vertex 4 512GB | Samsung | Samsung | Hitachi HD
| 830 512GB | 840 1TB | 1TB 5400 RPM
---------------------------------------------------------------------------------------------------
Encrypted BTRFS: | | |
reset_cache; time du -sh src | | |
514M src | | |
real 0.959s | 25.741s | 0.53s | 5.252s
| | |
Unencrypted btrfs | | |
mount -o compress=lzo,discard,nossd,space_cache,noatime | | |
reset_cache; time du -sh src kernel 3.5 | kernel 3.5 | kernel 3.11| kernel 3.5
514M src | | |
real 0.636s | 24.937s | 0.11s | 1.007s
| | |
Unencrypted ext4 | | |
mkfs.ext4 -O extent -b 4096 -E stride=128,stripe-width=128)| | |
mount -o discard,noatime /dev/sda2 /mnt/mnt2 | | |
reset_cache; time du -sh src | | |
519M src | | |
real 0.289s | 12.459s | 0.21s | 0.380s
Block Reads on Vertex 4 and Samsung 840 EVO (vs Samsung 830 and spinning hard drive)
Summary:
raw device speed is about the same for Samsung 830 and OCZ. The 840 is marginally faster
encrypted btrfs at 329MB/s is slower, but still is plenty fast. If anyone is doing software encryption faster than that, let me know :)
well, a bit over a year later, on the same laptop with the same CPU, encrypted btrfs block speed is about as bad as native block speed. Impressive.
unencrypted btrfs block speed is about as fast as block device speed in kernel 3.11. Very nice.
OCZ Vertex 4 512GB | Samsung | Samsung | Hitachi HD
| 830 512GB | 840 1TB | 1TB 5400 RPM
----------------------------------------------------------------------------------------------------------------
dd if=/dev/sda3 of=/dev/null bs=1M count=2000 | | |
2097152000 bytes (2.1 GB) copied, 4.65606 s, 450 MB/s | 479MB/s | 541MB/s | 104 MB/s
| | |
hdparm -t /dev/sda3 | | |
/dev/sda3: | | |
1430 MB in 3.00 seconds = 476.12 MB/s | 463.45 MB/s | 1112.81MB/s | 99.83 MB/s
| | |
Unencrypted ext4 block read: | | |
reset_cache; dd if=testcopy of=/dev/null bs=1M | | |
2146631680 bytes (2.1 GB) copied, 4.21942 s, 509 MB/s | | 458MB/s | 105 MB/s
| | |
Unencrypted btrfs block read: | | |
reset_cache; dd if=testcopy of=/dev/null bs=1M | | |
2146631680 bytes (2.1 GB) copied, 4.31774 s, 497 MB/s | 495 MB/s | 542MB/s | 90.3 MB/s
| | |
Encrypted btrfs block read: | | |
dd if=w2k-s001.vmdk of=/dev/null bs=1M kernel 3.5 | kernel 3.5 | kernel 3.11 | kernel 3.5
2146631680 bytes (2.1 GB) copied, 6.51863 s, 329 MB/s | 253 MB/sec | 533 MB/sec | 69.1 MB/s
Block writes on Vertex 4 and (vs spinning hard drive)
I didn't benchmark write speed on the Samsung 830 because I couldn't get random reads to be fast, so I didn't care about writes.
Summary:
my HD is slow enough (even though 95MB/s isn't actually slow) that encryption doesn't make it slower for block writes
on SSD however, it looks like I may hit an encryption bottleneck, speed goes down by almost 2x. Still 130MB/s is fast enough for writes, and I really mostly care about ultimate read speed, which I do have.
Somehow with kernel 3.11 on the Samsung 840 EVO, encrypted writes are now 80% of unencrypted speed. Very nice.
OCZ Vertex 4 512GB: | Samsung EVO | Hitachi 1TB
| 840 1TBM | 5400 RPM
---------------------------------------------------------------------------
Raw device write (write to /dev/sdxx bs=1M) | kernel 3.11 |
(2.1 GB) copied, 9.35569 s, 229 MB/s | 430 MB/s | 90.9 MB/s
| |
File write on unencrypted ext4: | kernel 3.11 |
(2.1 GB) copied, 11.3984 s, 188 MB/s | 495 MB/s | 92.0 MB/s
| |
File write on unencrypted btrfs: | kernel 3.11 |
(2.1 GB) copied, 8.58974 s, 250 MB/s | 413 MB/s | 130 MB/s
| |
File write through encryption layer on btrfs | kernel 3.11 |
(2.1 GB) copied, 16.4673 s, 130 MB/s | 341 MB/s | 95.6 MB/s
SSD winner and conclusions
SSDs are not hard drives. They are faster for sure when they work well, but don't expect them to fail like hard drives. They can fail suddenly with absolutely no warning, and chancees of recovery after that are slim to none. Recovery companies know how to deal with hard drive platters, but most aren't setup to deal with SSDs yet. Moral of the story: have backups (I did, and now I actually backup my SSD on the 2nd HDD in my laptop and can boot on the HDD if the SSD craps out while I'm on a trip).
The OCZ Vertex 4 512GB won by far obviously. The price is competitive, the speed beats pretty much everything out there, and the 5 year warranty shows that they mean business (most only offer 1, 2, or sometimes 3 years). 5 years is a record in this field.
Well, that was then, turns out OCZ was more confident than good, their drives did lock up, they made bad tradeoffs on other drives as explained by Luke on http://lkcl.net/reports/ssd_analysis.html, and they are now going bankrupt as of Dec 2013.
After about a year of use, I confirmed that the OCZ drive is just unstable with linux/dmcrypt/btrfs on my laptop. I'm not sure what part causes it to hang and require a power cycle every other day or so after a few months of use, but that was quite annoying.
I'm now hopeful that the Samsung EVO 840 will be my new "good drive" for a long longer :)
It sucks that it took me so long to find a good drive, and so many backup/restore cycles, but I think I got it now. Wish me luck! :) |
|