I’ve attached below my rough notes from Andy Bechtolsheim’s talk this morning at High Performance Transactions Systems 2009. The agenda for HPTS 2009 is up at: http://www.hpts.ws/agenda.html.

Andy is a founder of Sun Microsystems and of Arista Networks and is an incredibly gifted hardware designer. He’s consistently able to innovate and push the design limits while delivering reliable systems that really do work. It was an excellent talk. Unfortunately, he’s not planning to post the slides so you’ll have to make do with my rough notes below.

· Memory Technologies for Data Intensive Computing

· Speaker: Andy Bechtolsheim, Sun Microsystems

· Flash density is increasing at faster than Moore’s law and this is expected to continue

o Expect 100x improvement over the next 10 or 12 years

· Emerging technologies are coming

o Carbon Nano-Tube, Phase-change, Nano-ionic, …

o But new technologies take time so flash for now

· Expect: in 2022

o 64x cores but only 500W

o Would need 2.5 TB/s to memory and 250 GB/s to memory

· We would have been at 10GHz at 2022 but power density limits makes this impractical

o Power = clock * capacitance * Vdd^2

· Most saving will be packaging innovations: Multi-Chip 3D packaging (stacking cpu and many memory chips)

o More bandwidth through more channels without having to drive more pins (power issue)

· Expect no more memory per core than today and it could be worse

o Expect deeper multi-tier memories

· 10Gbps shipping today but expect 25GB in 2012

· Disks are SOOOOO slow in this context

o Forget disk for all but sequential and archival storage

· Sun Flash DIMM

o 30,000 Read IOPS, 20,000 writes

o Oracle did 7,717,510 tpmC using 24 sun flash devices

· Not easy to get 10^6 IOPS

o Limit is disk interface

o Answer is to go direct to PCI-X PCIe bus [jrh: this is what FusionIO does]

· Flash very different from DRAM: 100usec to read flash. About 1000x slower than DRAM.

· Enterprise flash coming:

o Rather than power optimized 33 Mhz transfers, run 133 Mhz or better

· Flash Summary:

o Expect the price of flash to ½ each year and the density to double each year

o Access times will fall by 50% per year

o Throughput will double each year

o Controllers are rapidly improving

o Interface moving from SATA to PCI-X PCIe

· Most promising new technologies are stacked chips (thu-Si via stacking) and flash

· Expect optic volumes to go up and price to go down driven by client side volumes:

o Intel Light Peak announced $5/client with on board chips

Generally Andy is incredibly positive on the continuation of Moors expects this pace of advancement to continue for at least another 10 years. He argues that disk is only useful for cold and sequential workloads and that flash is the future. Phase Change Memory and other new technologies may eventually replace flash but he points out these changes always take longer than predicted.

Expect flash to stay strong and relevant for the near term and expect it to be PCI-X PCIe connected rather than SATA attached.

James Hamilton

e: jrh@mvdirona.com

w: http://www.mvdirona.com

b: http://blog.mvdirona.com / http://perspectives.mvdirona.com