NVMe SSD Speed

[RonSwanson]

It's about 20% faster. This makes a difference for me when processing files that are a couple terabytes in size.


It doesn't. That assumes a uniform distribution for time-to-failure, which is incorrect.

What is the "correct" distribution for time to failure?

 

[cguy]

What is the "correct" distribution for time to failure?

Almost certainly some variation of a Bathtub curve, with a vanishingly small early mortality component on the left side (otherwise many people would be returning their laptops/storage devices almost as soon as they got it).

What is indeed true with RAID0, is that I am doubling my chance of failure vs a single drive every day, however, the odds of either of them failing very early is very low.

To take it too the limit - assume that drives all fail at exactly 10 years. In this case, there is no disadvantage from RAID0 at all. Now relax that a little bit and assume that there is a 10% chance of failure in year 9, and still a 100% chance of failure at year 10. This means that RAID0 is riskier, but only after 9 years, and then only a bit riskier in year 9. Now smooth that out, and you have a curve that will probably be near 0 for many years, before growing quickly after a good number of years.

Perhaps put another way, it’s like saying chewing gum doubles the probability of getting some disease that very rarely affects people under 80.

 

[ponder]

It's about 20% faster. This makes a difference for me when processing files that are a couple terabytes in size.

Fair enough as you fit the bill but be aware that your data is not safe unless you've implemented other safeguards.

 

[RonSwanson]

Almost certainly some variation of a Bathtub curve, with a vanishingly small early mortality component on the left side (otherwise many people would be returning their laptops/storage devices almost as soon as they got it).

Disappointing. I was hoping for some solid research from you to back it up

What is indeed true with RAID0, is that I am doubling my chance of failure vs a single drive every day, however, the odds of either of them failing very early is very low.

Agree with the first part in bold, which was my original point.

I do not agree with the second half of your sentence, because your "bathtub curve", if true, starts high, which contradicts it. I am inclined to accept your bathtub curve because, in my experience, PC electronics fail within the first 48 - 96 hours, which is why burn-in is so important, and why failure is more likely in that period.

I also concede that the M in MTBF stands for Mean, when I should have referred to Average, but then again no-one would have understood it, so I used the industry standard term describing failure rate. At least we agree on doubling the chances of failure, which, in retrospect, I should have used.

I am more interested in the 20% gain in speed, because it does not make too much sense when using NVMe drives. Do you have benchmarking to support it?

I also feel an overwhelming desire now to create 3-6 RAM drives and mount them in a RAID0 array to prove my hypothesis. Damm you!

 

[RonSwanson]

Fair enough as you fit the bill but be aware that your data is not safe unless you've implemented other safeguards.

Yep.
Or if the data is worthless, or can easily be recreated.

 

[cguy]

Disappointing. I was hoping for some solid research from you to back it up

It's a pretty standard model used for failures in engineering:

https://www.mfpt.org/wp-content/uploads/2019/10/Heiser-Thomas-Bathtub-Failure-Distribution-MTBF-Paper.pdf

Bathtub curve - Wikipedia

en.wikipedia.org

Agree with the first part in bold, which was my original point.

That's not the same as halving your MTBF.

I do not agree with the second half of your sentence, because your "bathtub curve", if true, starts high, which contradicts it. I am inclined to accept your bathtub curve because, in my experience, PC electronics fail within the first 48 - 96 hours, which is why burn-in is so important, and why failure is more likely in that period.

As per my comment "vanishingly small early mortality component on the left side", the left side of the NVME Bathtub curve, has either being dampened by engineering effort is not exposed to the consumer (e.g., by screening and/or burn-in testing), so that isn't a concern.

I also concede that the M in MTBF stands for Mean, when I should have referred to Average, but then again no-one would have understood it, so I used the industry standard term describing failure rate. At least we agree on doubling the chances of failure, which, in retrospect, I should have used.

I am more interested in the 20% gain in speed, because it does not make too much sense when using NVMe drives. Do you have benchmarking to support it?

Well, it's about 20% 60% faster than the peak speed of the NVMEs being used, so I expect I am seeing a 20%+ 60%+ performance boost in sequential reads.

I also feel an overwhelming desire now to create 3-6 RAM drives and mount them in a RAID0 array to prove my hypothesis. Damm you!

I did find some benchmarks online before buying, and more than 2 didn't benefit. Not to surprising, given that even with 2 drives, the scaling isn't linear here. Certainly hitting a downstream limit, although I haven't bothered to figure out where exactly.

Looking at some of the other parameters, I doubled the queue depth, and got even better utilization:



I do agree that for most users, it's probably not going to make a whole lot of difference, but for me it can make a difference of several minutes per run for the big data sets. There's also a risk of RAID0 negatively affecting latency, but sequential reads are where I expect most of my human waiting time to be.

 

[cguy]

Yep.
Or if the data is worthless, or can easily be recreated.

It's toy data that I can recreate (I don't use real data at home), but I need it to be consistent so that I get run-to-run consistency for regression testing. I doubt that this is going to be significantly less reliable than a single NVME, however, no local storage is going to fit the bill for keeping things I need to keep.

 

[RonSwanson]

It's a pretty standard model used for failures in engineering:

https://www.mfpt.org/wp-content/uploads/2019/10/Heiser-Thomas-Bathtub-Failure-Distribution-MTBF-Paper.pdf

Bathtub curve - Wikipedia

en.wikipedia.org

Yes, I wasn't referring to the model (which everyone is familiar with), but independent tests / data on NVMe drives. "almost certainly" is your opinion based on the model. Most probably correct is Bad science.

Forget about it though.

That's not the same as halving your MTBF.

I already conceded that point.

As per my comment "vanishingly small early mortality component on the left side", the left side of the NVME Bathtub curve, has either being dampened by engineering effort is not exposed to the consumer (e.g., by screening and/or burn-in testing), so that isn't a concern.

Au contraire. This is very often "exposed to the consumer", which explains the high rate of RMA components that occur in the first few days after purchase, it depends on the vendor's factory quality control. Moreover, many like me buy components and therefore burn-in happens after assembly. Were the two NVME drives in your screenshot (which I assume were yours) received as "burned-in"?

Well, it's about 20% 60% faster than the peak speed of the NVMEs being used, so I expect I am seeing a 20%+ 60%+ performance boost in sequential reads.


I did find some benchmarks online before buying, and more than 2 didn't benefit. Not to surprising, given that even with 2 drives, the scaling isn't linear here. Certainly hitting a downstream limit, although I haven't bothered to figure out where exactly.

Interesting, because with mechanical HDDs performance does scale linearly when adding more drives. Up to a point where the latency of the drive seek (positioning of the head's actuator) is perfectly interleaved with the array controller's write speed, and then levels out.
The fact that more than two NVMes provides no additional performance gain could therefore be predicted, what I am interested in is why two provide greater performance than just one.

Looking at some of the other parameters, I doubled the queue depth, and got even better utilization:

View attachment 1676703

I do agree that for most users, it's probably not going to make a whole lot of difference, but for me it can make a difference of several minutes per run for the big data sets. There's also a risk of RAID0 negatively affecting latency, but sequential reads are where I expect most of my human waiting time to be.

Thanks, enjoy.

 

[cguy]

Yes, I wasn't referring to the model (which everyone is familiar with), but independent tests / data on NVMe drives. "almost certainly" is your opinion based on the model. Most probably correct is Bad science.

"almost certainly" is just what I would expect the distribution to look like. The "halving MTBF" statement assumes a uniform distribution, which would assume that wear and tear had no impact, which is clearly false.

Au contraire. This is very often "exposed to the consumer", which explains the high rate of RMA components that occur in the first few days after purchase, it depends on the vendor's factory quality control. Moreover, many like me buy components and therefore burn-in happens after assembly. Were the two NVME drives in your screenshot (which I assume were yours) received as "burned-in"?

I seriously doubt that it's very often or a high rate. Do you have data to back that up?

Yes, those are my NVMEs on my Alienware laptop.

Interesting, because with mechanical HDDs performance does scale linearly when adding more drives. Up to a point where the latency of the drive seek (positioning of the head's actuator) is perfectly interleaved with the array controller's write speed, and then levels out.
The fact that more than two NVMes provides no additional performance gain could therefore be predicted, what I am interested in is why two provide greater performance than just one.

PCIe Gen 4.0 x4 Unidirectional bandwidth is 8GB/s, so (assuming there isn't an x4 switch in front of the x4 ports), you would need two of these to exceed 8GB/s, for one. Also, most 4.0 x4 NVMEs don't saturate 8GB/s, so having two of them run in parallel, provides up to 2x the bandwidth of the individual drives theoretically. My NVMEs have a sticker read bandwidth of 7GB/s, so I'm getting 11GB/s of a theoretical 14GB/s.

 

[RonSwanson]

"almost certainly" is just what I would expect the distribution to look like. The "halving MTBF" statement assumes a uniform distribution, which would assume that wear and tear had no impact, which is clearly false.

Assumption is also bad science.
We already agreed that it doubles the probability of failure, no?

I seriously doubt that it's very often or a high rate. Do you have data to back that up?

Um, how about the paper you suggested I read?


You also skipped my question: " Were the two NVME drives in your screenshot (which I assume were yours) received as "burned-in"? "

 

[cguy]

Assumption is also bad science.
We already agreed that it doubles the probability of failure, no?

It sounds like you’re not getting that “doubles the probability of failure” does not mean that it significantly decreases the expected time to failure.

EDIT: After rereading this - I do agree that if there was a high failure rate at the start of use, two drives would indeed be much more risky, although I doubt that the failure rate is high enough to warrant concern.

Um, how about the paper you suggested I read?
View attachment 1676731

That’s just the model for the distribution. For something like consumer grade NVMEs, that circled bit is nearly flat. How many NVMEs do you think break a few days after they arrive in a laptop?

You also skipped my question: " Were the two NVME drives in your screenshot (which I assume were yours) received as "burned-in"? "

I have no idea - whatever Dell/Alienware do, is what I received.

 

[RonSwanson]

It sounds like you’re not getting that “doubles the probability of failure” does not mean that it significantly decreases the expected time to failure.

EDIT: After rereading this - I do agree that if there was a high failure rate at the start of use, two drives would indeed be much more risky, although I doubt that the failure rate is high enough to warrant concern.

That’s just the model for the distribution. For something like consumer grade NVMEs, that circled bit is nearly flat. How many NVMEs do you think break a few days after they arrive in a laptop?


I have no idea - whatever Dell/Alienware do, is what I received.

Thank you for your responses, it was an interesting engagement.