Author Archive
In When SSDs Make Sense in Server Applications, we looked at where Solid State Drives (SSDs) were practical in servers and services. On the client side, there are even more reasons to use SSDs and I expect that within three years, more than half of enterprise laptops will have NAND Flash as at least part…
In past posts, I’ve talked a lot about Solid State Drives. I’ve mostly discussed about why they are going to be relevant on the server side and the shortest form of the argument is based on extremely hot online transaction processing systems (OLTP). There are potential applications as reliable boot disks in blade servers and…
Albert Greenberg and I missed Hotnets 2008 last week due to a conflicting meeting down in California but Ken Church was there to present our On Delivering Embarrassingly Distributed Cloud Services paper. I summarized the paper in a recent blog entry: Embarrassingly Distributed Cloud Services and the abstract from the paper follows: Very large data…
Google has long enjoyed a reputation for running efficient data centers. I suspect this reputation is largely deserved but, since it has been completely shrouded in secrecy, that’s largely been a guess built upon respect for the folks working on the infrastructure team rather than anything that’s been published. However, some of the shroud of…
An interesting file system study is at this year’s USENIX Annual Technical Conference. The paper Measurement and Analysis of Large-Scale Network File System Workloads looks at CIFS remote file system access patterns from two populations. The first a large file store of 19TB serving 500 software developers and the second a medium sized file store…
Ken Church, Albert Greenberg, and I just finished On Delivering Embarrassingly Distributed Cloud Services which has been accepted for presentation at ACM Hotnets 2008 in Calgary, Alberta October 6th and 7th. This paper followed from the discussion and debate around a blog entry that Ken and I did some time back: Diseconomies of scale where…
Earlier today, I gave a talk at LADIS 2008 (Large Scale Distributed Systems & Middleware) in Yorktown Heights, New York. The program for LADIS is at: http://www.cs.cornell.edu/projects/ladis2008/program.html. The slides presented are posted to: http://mvdirona.com/jrh/TalksAndPapers/JamesRH_Ladis2008.pdf. The quick summary of the talk: Hosted services will be a large part of enterprise information processing and consumer services with…
This note describes a conversation I’ve had multiple times with data center owners and concludes that blade servers frequently don’t help and they sometimes hurt, easy data center power utilization improvements are available independent of the blade server premium, and enterprise data center owners have a tendency to buy gadgets from the big suppliers rather…
IBM just announced achieving over one million Input-output operations per second: IBM Breaks Performance Records Through Systems Innovation. That’s an impressive number. To put the achievement in context, a very good (and way too expensive) enterprise disk will deliver somewhere between 180 to just over 200 IOPS. A respectable, but commodity, SATA disk will usually…
In Designing and Deploying Internet Scale Services I’ve argued that all services should expect to be overloaded and all services should expect mass failures. Very few do and I see related down-time in the news every month or so. The Windows Genuine Advantage failure (WGA Meltdown…) from a year ago is a good example in…
Facebooks F8 conference was held last month in San Francisco. During his mid-day keynote Mark Zuckerberg reported that the Facebook platform now has 400,000 developers and 90 million users of which 32% are from the United States. The platforms US user population grew 2.4x last year while the international population grew at an astounding 5.1x….
Kevin Clark, Director of IT Operations at Lucasfilm was interviewed by On-Demand Enterprise in We’ve Come a Long Way Since Star Wars. His organization owns IT for LucasArts, Lucasfilm, and Industrial Light and Magic. Lucasfilm runs a 4,500 server dedicated rendering farm and they expand this farm with workstations when they are not in use…
Last Friday I arrived back from vacation (Back from the Outside Passage in BC) to 3,600 email messages. I’ve been slogging through them through the weekend to now and I’m actually starting to catch up. Yesterday Tom Kleinpeter pointed me to this excellent posting from Jason Sobel of Facebook: Scaling Out. This excellent post describes…
Going boating: http://mvdirona.com/ so I’ll be taking a break from blogging until mid-august when I’m back and caught back up. Enjoy, –jrh James Hamilton, Data Center FuturesBldg 99/2428, One Microsoft Way, Redmond, Washington, 98052 W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 | JamesRH@microsoft.com H:mvdirona.com | W:research.microsoft.com/~jamesrh | blog:http://perspectives.mvdirona.com
I’ve been collecting scaling stories for some time now and last week I came across the following run down on Fliker scaling: Federation at Flickr: Doing Billions of Queries Per Day by Dathan Vance Pattishall, the Flickr database guy. The Flickr DB Architecture is sharded with a PHP access layer to maintain consistency. Flickr users…
I just got back from O’Reilly’s Foo Camp. Foo is an interesting conference format in that there is no set agenda. It’s basically self organized as a open space-type event but that’s not what makes it special. What makes Foo a very cool conference is the people. Lots of conferences invite good people but few…
In When SSDs Make Sense in Server Applications, we looked at where Solid State Drives (SSDs) were practical in servers and services. On the client side, there are even more reasons to use SSDs and I expect that within three years, more than half of enterprise laptops will have NAND Flash as at least part…
In past posts, I’ve talked a lot about Solid State Drives. I’ve mostly discussed about why they are going to be relevant on the server side and the shortest form of the argument is based on extremely hot online transaction processing systems (OLTP). There are potential applications as reliable boot disks in blade servers and…
Albert Greenberg and I missed Hotnets 2008 last week due to a conflicting meeting down in California but Ken Church was there to present our On Delivering Embarrassingly Distributed Cloud Services paper. I summarized the paper in a recent blog entry: Embarrassingly Distributed Cloud Services and the abstract from the paper follows: Very large data…
Google has long enjoyed a reputation for running efficient data centers. I suspect this reputation is largely deserved but, since it has been completely shrouded in secrecy, that’s largely been a guess built upon respect for the folks working on the infrastructure team rather than anything that’s been published. However, some of the shroud of…
An interesting file system study is at this year’s USENIX Annual Technical Conference. The paper Measurement and Analysis of Large-Scale Network File System Workloads looks at CIFS remote file system access patterns from two populations. The first a large file store of 19TB serving 500 software developers and the second a medium sized file store…
Ken Church, Albert Greenberg, and I just finished On Delivering Embarrassingly Distributed Cloud Services which has been accepted for presentation at ACM Hotnets 2008 in Calgary, Alberta October 6th and 7th. This paper followed from the discussion and debate around a blog entry that Ken and I did some time back: Diseconomies of scale where…
Earlier today, I gave a talk at LADIS 2008 (Large Scale Distributed Systems & Middleware) in Yorktown Heights, New York. The program for LADIS is at: http://www.cs.cornell.edu/projects/ladis2008/program.html. The slides presented are posted to: http://mvdirona.com/jrh/TalksAndPapers/JamesRH_Ladis2008.pdf. The quick summary of the talk: Hosted services will be a large part of enterprise information processing and consumer services with…
This note describes a conversation I’ve had multiple times with data center owners and concludes that blade servers frequently don’t help and they sometimes hurt, easy data center power utilization improvements are available independent of the blade server premium, and enterprise data center owners have a tendency to buy gadgets from the big suppliers rather…
IBM just announced achieving over one million Input-output operations per second: IBM Breaks Performance Records Through Systems Innovation. That’s an impressive number. To put the achievement in context, a very good (and way too expensive) enterprise disk will deliver somewhere between 180 to just over 200 IOPS. A respectable, but commodity, SATA disk will usually…
In Designing and Deploying Internet Scale Services I’ve argued that all services should expect to be overloaded and all services should expect mass failures. Very few do and I see related down-time in the news every month or so. The Windows Genuine Advantage failure (WGA Meltdown…) from a year ago is a good example in…
Facebooks F8 conference was held last month in San Francisco. During his mid-day keynote Mark Zuckerberg reported that the Facebook platform now has 400,000 developers and 90 million users of which 32% are from the United States. The platforms US user population grew 2.4x last year while the international population grew at an astounding 5.1x….
Kevin Clark, Director of IT Operations at Lucasfilm was interviewed by On-Demand Enterprise in We’ve Come a Long Way Since Star Wars. His organization owns IT for LucasArts, Lucasfilm, and Industrial Light and Magic. Lucasfilm runs a 4,500 server dedicated rendering farm and they expand this farm with workstations when they are not in use…
Last Friday I arrived back from vacation (Back from the Outside Passage in BC) to 3,600 email messages. I’ve been slogging through them through the weekend to now and I’m actually starting to catch up. Yesterday Tom Kleinpeter pointed me to this excellent posting from Jason Sobel of Facebook: Scaling Out. This excellent post describes…
Going boating: http://mvdirona.com/ so I’ll be taking a break from blogging until mid-august when I’m back and caught back up. Enjoy, –jrh James Hamilton, Data Center FuturesBldg 99/2428, One Microsoft Way, Redmond, Washington, 98052 W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 | JamesRH@microsoft.com H:mvdirona.com | W:research.microsoft.com/~jamesrh | blog:http://perspectives.mvdirona.com
I’ve been collecting scaling stories for some time now and last week I came across the following run down on Fliker scaling: Federation at Flickr: Doing Billions of Queries Per Day by Dathan Vance Pattishall, the Flickr database guy. The Flickr DB Architecture is sharded with a PHP access layer to maintain consistency. Flickr users…
I just got back from O’Reilly’s Foo Camp. Foo is an interesting conference format in that there is no set agenda. It’s basically self organized as a open space-type event but that’s not what makes it special. What makes Foo a very cool conference is the people. Lots of conferences invite good people but few…