It’s not (just) about performance

Whenever I see a huge performance number without the denominator, I shake my head. It’s easy to get a big performance number on almost any dimension but what is far more difficult is getting a great work done per dollar. Performance alone is not interesting.

I’m super interested in flash SSDs and see great potential for SSDs in both client and server-side systems. But, our industry is somewhat hype driven. When I first started working with SSDs and their application to server workloads, many thought it was a crazy ideas pointing out that the write rates were poor and they would wear out in days. The former has been fixed in Gen 2 devices and later was never true. Now SSDs are climbing up the hype meter and I find myself arguing on the other side: they don’t solve all problems. I still see the same advantages I saw before but I keep seeing SSDs proposed for applications where they simply are not the best price/performing solution.

Rather than write the article about where SSDs are a poor choice, I wrote two articles on where they were a good one:

· When SSDs Make Sense in Server Applications

· When SSDs Make Sense in Client Applications

SSDs are really poor choices for large sequential workloads. If you want aggregate sequential bandwidth, disks deliver it far cheaper.

In this article and referenced paper (Microslice Servers), I argue in more detail why performance is a poor measure for servers on any dimension. It’s work done per dollar and work done per watt we should be measuring.

I recently came across a fun little video, Samsung SSD Awesomeness. It’s actually a Samsung SSD advertisement. Overall, the video is fun. It’s creative and sufficiently effective that I watched the entire thing and you might as well. Clearly it’s a win for Samsung. However, the core technical premise is broken. What they are showing is that you can get 2 GB/s by RAID 24 SSDs together. This is unquestionably true. However, we can get 2 GB/s by raiding together 17 Seagate Barracuda 7200.11 (big, cheap, slow hard drives) at considerably lower cost. The 24 SSDs will produce awe striking random I/O performance and not particularity interesting sequential performance. 24 SSDs is not the cheapest way to get 2GB/s of sequential I/O.

Both Samsung and Intel have excellent price performing SSDs and both can produce great random IOPS/$. There are faster SSDs out there (e.g. FusionIO) but the Samsung and Intel components are better price/performers and that’s the metric that really matters. However, none of them are good price/performers on pure sequential workloads and yet that’s how I see them described and that’s the basis for many purchasing decisions.

See Annual Fully Burdened Cost of Power for a quick analysis of when an SSD can be a win based upon power savings and IOPS/$.

Conclusion: If the workload is large and sequential, use a hard disk drive. If it’s hot and random, consider an SSD-based solution.

-jrh

Thanks to Sean James of Microsoft for sending the video my way.

James Hamilton, Amazon Web Services

1200, 12th Ave. S., Seattle, WA, 98144
W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 |
james@amazon.com

H:mvdirona.com | W:mvdirona.com/jrh/work | blog:http://perspectives.mvdirona.com

3 comments on “It’s not (just) about performance
  1. Eas, I did see that the 1,000 Watt supply but missed that they actually used two! Good catch. 2,000W of PSU is pretty extreme.

    Yaron, I agree that perf charts comparing against current technology is very helpful. And, perf charts without costs are useful. I have no ideas what to do with fast without know the cost.

    –jrh
    jrh@mvdirona.com

  2. Yaron Bar-Or says:

    It would be great if such news could indeed be put into context. For example, it would be interesting to see a graphic comparison of the new solution to existing ones. The factblender.com hard drive chart takes a step in that direction — it shows capacity vs. price. In this case, it would also be interesting to see dimensions such as throughput, latency, and power consumption compared.

  3. eas says:

    As for the perf-per-watt of that samsung, um, solution, did you notice that they used two 1000W power supplies? I’m sure it was just for show, or because they couldn’t be bothered to get some Y adapters to provide enough power taps, or both.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.