IBM just announced achieving over one million Input-output operations per second: IBM Breaks Performance Records Through Systems Innovation. That’s an impressive number. To put the achievement in context, a very good (and way too expensive) enterprise disk will deliver somewhere between 180 to just over 200 IOPS. A respectable, but commodity, SATA disk will usually drive somewhere in the 70 to 100 IOPS range.

To achieve this goal, IBM actually used a Fusion-IO NAND flash based storage component. It’s unfortunate that the original press release from IBM didn’t include FusionIO. However, an excellent blog write-up on the performance run by Barry Whyte of IBM offers the details behind the effort: 1M IOPs from Flash – actions speak louder than words. The Fusion-IO press release is at: Fusion-io and IBM Team to Improve Enterprise Storage Performance.

FusionIO is a PCIe storage subsystem based upon NAND flash. I mentioned them in 100,000 IOPS. It’s a bit expensive at this point but a very nice part nonetheless. NAND prices continue to free-fall based upon mammoth volumes driven by usage in consumer devices and some over-capacity in the NAND market. As the base technology prices fall and sales of enterprise Flash-based storage devices increases, I expect we’ll see pricing improvements as well over the near term. And, for the very hottest online transaction workloads where IOPS are the primary limiting factor, even current prices work and we’re starting to see some high I/O rate workloads migrate from spinning media to NAND flash. Some have already moved and I know of many more that have devices in test.

Digging deeper into the IBM result, we see that the Fusion-IO part in this run was mounted behind a SAN. I’ve already taken a bit of heat on this point as it’s well known that I’m not a lover of SANs. Actually, its not really true that I hate SANs. What I hate are expensive, scale-up solutions and it is true that many SAN fall into this catagory. I want servers, storage, and networking to all be built from clusters of commodity components. Quarter million dollar network routers just don’t make sense to me and most SANs are not affordable at internet service scale. Essentially, high end network routers and SAN storage arrays are the last bastion of the mainframe — very high quality, very expensive, scale-up solutions. As an example, consider the Symmetrix DMX3000. At full scale, it has 576 disk drives, ¼ TB of memory and over 100 1GHz embedded PowerPC processors. When it was announced back in 2003, the starting price was $1.7M (in lightly configured form– the sky is the limit).

It’s really mainframe priced storage subsystems that I’m objecting to. SANs could be great if built from commodity parts and priced to sell in volume. The ability to migrate storage between machines is clearly useful. I’m not in love with an entire networking and switching infrastructure dedicated to storage (Fibre Channel) but that’s not inheriently required by SANs either. FCOE should solve that problem and iSCSI does.

The IBM Million IOPS number built upon Fusion-IO NAND Flash components and a virtual SAN over a cluster of Intel-based servers is very interesting.

–jrh

James Hamilton, Data Center Futures
Bldg 99/2428, One Microsoft Way, Redmond, Washington, 98052
W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 | JamesRH@microsoft.com

H:mvdirona.com | W:research.microsoft.com/~jamesrh | blog:http://perspectives.mvdirona.com