There is no question that cloud computing is going to a big part of the future of server-side systems. What I find interesting is the speed with which this is happening. Look at recent network traffic growth rates from AWS:
From: http://aws.typepad.com/aws/2008/05/lots-of-bits.html
AWS is now consuming considerably more bandwidth than Amazon’s global web sites. Phenomenal growth and impressive absolute size.
Continuing to look at growth, I saw a chart a few weeks back on the Amazon Web Services Blog that illustrates the value of a pay-as-you-go and pay-as-your-grow service. This chart shows the number of EC2 servers in use by Animoto over a couple of week period. Note the explosion in EC2 server usage in the three day period from 4/15 through 4/18 and imagine trying to do capacity planning for Animoto. They went from roughly 50 servers to needing more than 3,500 in three days. Imagine having to predict growth and get servers racked, stacked and online in time to meet the growth. Nearly impossible.
From: http://aws.typepad.com/aws/2008/04/animoto—scali.html (Emre Kiciman sent it my way).
When you next hear “why web services?”, think of this chart.
Another point I hear frequently around web services is, “sure, they are used by start-ups but REAL enterprises would never use them due to security and data privacy reasons.” Again, utter bunk but it’s a frequently repeated quip. I led the Exchange Hosted Services team and we provided hosted email anti-malware and archiving. The service was originally targeting small and medium sized businesses and many from those categories did use it. But, what was interesting was the number of name-brand, world-wide enterprises that recognized the cost and quality advantages of using hosting services. Valuable internal enterprise resources are best saved for tasks that add value to the business.
Perhaps the large enterprises will use hosted email services but what about low level services such as EC2 and S3? Again, it’s the same story. If the value is there, companies of all sizes will use it. From the Amazon 4th quarters earnings call, TechCrunch reports (Who Are The Biggest Users of Amazon Web Services? It’s Not Startups):
So who are using these services? A high-ranking Amazon executive told me there are 60,000 different customers across the various Amazon Web Services, and most of them are not the startups that are normally associated with on-demand computing. Rather the biggest customers in both number and amount of computing resources consumed are divisions of banks, pharmaceuticals companies and other large corporations who try AWS once for a temporary project, and then get hooked.
Big companies are jumping in as well.
Google recently entered the cloud computing market with Google Application Engine. They are only a couple months in beta and report they have allowed in 60,000 developers in that short period of time. The amazing thing is the apparent size of the back log. The forums are full of people complaining that they can’t yet get on (Sriram Krishnan sent my way).
Wired recently published “Cloud Computing. Available at Amazon.com Today”.
It’s unusual for a new model to grow so fast and it’s close to unprecedented to see so much early growth in the enterprise. However, when the potential savings are this large, big things can happen.
–jrh
James Hamilton, Windows Live Platform Services
Bldg RedW-D/2072, One Microsoft Way, Redmond, Washington, 98052
W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 | JamesRH@microsoft.com
H:mvdirona.com | W:research.microsoft.com/~jamesrh | blog:http://perspectives.mvdirona.com
I agree.
On the capacity planning point, I’ve become convinced that you simply can’t do reliable capacity planning for a single service. You could get Animoto’ed at any time. The best defense is to do capacity planning for heterogeneous set of properties. The spikes and valleys are not highly correlated and the growth rates are smoothed.
With service aggregations, reliable capacity planning is hard but possible. With a single service, the unpredictable events drown out the data.
–jrh
i’d be really interested in knowing how capacity planning for something like AWS/EC2/S3/GAE would work. i’d imagine customers tend to grow and evolve at many different speeds and scales, and while peak capacity (plus room to grow) probably is a little easier to pinpoint time-wise (that would be my guess for these guys), i’d imagine it’s probably a lot more difficult to predict and plan data center growth.
then again, for both google and amazon, this may be less of an issue than it would be for a company that *only* does cloud computing – google and amazon both have huge data centers and can merge cloud computing service capacity planning with core service capacity planning, as well as enjoy a little overlap and admission control / load shedding across services.
all this being said, i’m incredibly excited about both of these offerings. as fun as it is to build-your-own data center/cloud infrastructure, when you’ve got amazon / google there to do it for you, you’ve got to be criminally negligent as a CTO/leader in a tech business to not at least strongly consider those offerings as opposed to building your own (which would likely be bound to failure anyway). incredibly interesting and totally mind-blowing how fast this is beginning to take off out in the wild… i’ve believed for almost a year now that goog/amzn offerings like these are really a major gift to innovators and startups, leveling the playing field for those guys to build some incredible stuff without having to invest in building infrastructure. brilliant.