Archive For The “Services” Category
Back in the early 90’s I attended High Performance Transactions Systems for the first time. I loved it. It’s on the ocean just south of Monterey and some of the best in both industry and academia show up to attend the small, single tracked conference. It’s invitational and kept small so it can be interactive….
Earlier today I presented Where Does the Power Go and What to do About it at the Western Washington Chapter of AFCOM. I basically presented the work I wrote up in the CIDR paper: The Case for Low-Cost, Low-Power Servers. The slides are at: JamesHamilton_AFCOM2009.pdf (1.22 MB). The general thesis of the talk is that…
Service billing is hard. It’s hard to get invoicing and settlement overheads low. And billing is often one of the last and least thought of components of a for-fee online service systems. Billing at low overhead and high scale takes engineering and this often doesn’t get attention until after the service beta period. During a…
Patterson, Katz, and the rest of the research team from Berkeley have an uncanny way of spotting a technology trend or opportunity early. Redundant Array of Inexpensive Disk (RAID) and Reduced Instruction Set Computing (RISC) are two particularly notable research contributions from this team amongst numerous others. Yesterday, the Berkeley Reliable, Adaptable, Distributed Systems Lab…
Yesterday, IBM announced it is offering access to IBM Software in the Amazon Web Services Cloud. IBM products now offered for use in the Amazon EC2 environment include: DB2 Express-C 9.5 Informix Dynamic Server Developer Edition 11.5 WebSphere Portal Server and Lotus Web Content Management Standard Edition WebSphere sMash The IBM approach to utility computing…
Over the years, I’ve noticed that most DoS attacks are actually friendly fire. Many times I’ve gotten calls from our Ops Manager saying the X data center is under heavy attack and we’re rerouting traffic to the Y DC only later to learn that the “attack” was actually a mistake on our end. There is…
Microsoft has announced the delay of Chicago and Dublin earlier this week (Microsoft will open Dublin and Chicago Data Centers as Customer Demand Warrants. A few weeks ago the Des Moines data center delay was announced (http://www.canadianbusiness.com/markets/market_news/article.jsp?content=D95T2TRG0). Arne Josefsberg and Mike Manos announced these delays in there Building a Better Mousetrap, a.k.a. Optimizing for Maximum…
I’ve long argued that tough constraints often make for a better service and few services are more constrained than Wikipedia where the only source of revenue is user donations. I came across this talk by Domas Mituzas of Wikipedia while reading old posts on Data Center Knowledge. The posting A Look Inside Wikipedia’s Infrastructure includes…
From Viraj Mody of the Microsoft Live Mesh team sent this my way: Dan Farino About MySpace Architecture. MySpace, like Facebook, uses relational DBs extensively front-ended by a layer of Memcached servers. Less open source at MySpace but otherwise unsurprising – a nice scalable design with 3000 front end servers with well over 100 database…
I wrote this blog entry a few weeks ago before my recent job change. It’s a look at the cost of high-scale storage and how it has fallen over the last two years based upon the annual fully burdened cost of power in a data center and industry disk costs trends. The observations made in…
Resource Consumption Shaping is an idea that Dave Treadwell and I came up with last year. The core observation is that service resource consumption is cyclical. We typically pay for near peak consumption and yet frequently are consuming far below this peak. For example, network egress is typically charged at the 95th percentile of peak…
In the Cost of Power in Large-Scale Data Centers, we looked at where the money goes in a large scale data center. Here I’m taking similar assumptions and computing the Annual Cost of Power including all the infrastructure as well as the utility charge. I define the fully burdened cost of power to be the…
Michael Manos yesterday published Our Vision for Generation 4 Modular Data Centers – One Way of Getting it Just Right. In this posting, Mike goes through the next generation modular data center designs for Microsoft. Things are moving quickly. I first argued for modular designs in a Conference on Innovative Data Systems paper submitted in…
Ed Lazowska of University of Washington invited me in speak to his CSE 490H class. This is a great class that teaches distributed systems in general and the programming assignments are MapReduce workloads using Hadoop. I covered two major topics, the first on high scale service best practices. How to design, develop, and efficiently operate…
Yesterday, AWS announced new pricing for SimpleDB and its noteworthy: free developer usage for 6 months. No charge for up to 1GB of ingres+egress, 25 machine hours, and 1GB storage. To help you get started with Amazon SimpleDB, we are providing a free usage tier for at least the next six months. Each month, there…
In a comment to the last blog entry, Cost of Power in Large-Scale Data Centers Doug Hellmann brought up a super interesting point It looks like you’ve swapped the “years” values from the Facilities Amortization and Server Amortization lines. The Facilities Amortization line should say 15 years, and Server 3. The month values are correct,…