I wrote this blog entry a few weeks ago before my recent job change. It’s a look at the cost of high-scale storage and how it has fallen over the last two years based upon the annual fully burdened cost of power in a data center and industry disk costs trends. The observations made in this post are based upon understanding these driving costs and should model any efficient, high-scale bulk storage farm. But, and I need to be clear on this, it was written prior to my joining AWS and there is no information below relating to any discussions I’ve had with AWS or how the AWS team specifically designs, deploys, or manages their storage farm.
When Amazon released Amazon S3, I argued that it was priced below cost at $1.80/GB/year. At that time, my estimate of their cost was $2.50/GB/year. The Amazon charge of $1.80/GB/year for data to be stored twice in each of two data centers is impressive. It was amazing when it was released and it remains an impressive value today.
Even though the storage price was originally below cost by my measure, Amazon could still make money if they were running a super-efficient operation (likely the case). How could they make money charging less than cost for storage? Customers are charged for ingress/egress on all data entering or leaving the AWS cloud. The network ingress/egress charged by AWS are reasonable, but telecom pricing strongly rewards volume purchases, so what Amazon pays is likely much less than the AWS ingress/egress charges. This potentially allows the storage business to be profitable even when operating at a storage cost loss.
One concern I’ve often heard is the need to model the networking costs between the data centers since there are actually two redundant copies stored in two independent data centers. Networking, like power, is usually billed at the 95 percentile over a given period. The period is usually a month but more complex billing systems exist. The constant across most of these high-scale billing systems is that the charge is based upon peaks. What that means is adding ingress or egress at an off peak time is essentially free. Assuming peaks are short-lived, the sync to the other data center can be delayed until the peak has passed. If the SLA doesn’t have a hard deadline on when the sync will complete (it doesn’t), then the inter-DC bandwidth is effectively without cost. I call this technique Resource Consumption Shaping and it’s one of my favorite high-scale service cost savers.
What is the cost of storage today in an efficient, commodity bulk-storage service? Building upon the models in the cost of power in large-scale data centers and the annual fully burdened cost of power, here’s the model I use for cold storage with current data points:
Note that this is for cold storage and I ignore the cost of getting the data to or from the storage farm. You need to pay for the networking you use. Again, since it’s cold storage the model assume you can use 80% of the disk which wouldn’t be possible for data with high I/O rates per GB. And we’re using commodity SATA disks at 1TB that only consume 10W of power. This is a cold storage model. If you are running higher I/O rates, figure out what percentage of the disk you can successfully use and update the model in the spreadsheet (ColdStorageCost.xlsx (13.86 KB)). If you are using higher-power, enterprise disk, you can update the model to use roughly 15W for each.
Update: Bryan Apple found two problems with the spreadsheet that have been updated in the linked spreadsheet above. Ironically the resulting fully brudened cost/GB/year is the unchanged. Thanks Bryan.
For administration costs, I’ve used a fixed, fairly conservative factor of a 10% uplift on all other operations and administration costs. Most large-scale services are better than this and some are more than twice as good but I included the conservative 10% number
Cold storage with 4x copies at high-scale can now be delivered at: 0.80/GB/year. It’s amazing what falling server prices and rapidly increasing disks sizes have done. But, it’s actually pretty hard to do and I’ve led storage related services that didn’t get close to this efficient -- I still think that Amazon S3 is a bargain.
Looking at the same model but plugging in numbers from about two years ago shows how fast we’re seeing storage costs plunge. Using $2,000 servers rather than $1,200, server power consumption at 250W rather than 160W, disk size at ½ TB and disk cost at $250 rather than 160, yield an amazingly different $2.40/GB/year.
Cold storage with redundancy at: $0.80 GB/year and still falling. Amazing.
Disclaimer: The opinions expressed here are my own and do not
necessarily represent those of current or past employers.