August 25th, 2021 marks the 15-year anniversary for EC2. Contemplating the anniversary has me thinking back to when I first got involved with cloud-hosted services. It was back in early 2005, about a year before S3 was announced, and I was at a different company working on a technical due diligence project for a corporate acquisition. I’ve always enjoyed assessing startups. It’s a challenge to dive in with limited time and both learn how smart engineers have innovated to solve interesting problems and also try to find any technical blind spots that may have slipped past.
In this 2005 example, the company under review was a cloud-hosted service. We decided to go ahead with the acquisition, it became a wholly-owned subsidiary, and I took over as General Manger. Operating this early cloud service taught me two lessons fairly quickly:
- First, customers just loved the cloud-hosted service model and could adopt services quickly;
- Second, it was very clear to me that we were only just getting started. The service I was working on had many flaws, there were availability issues, and we were delivering nothing close to what was possible. There looked to be at least a 10x improvement possible in both cost and in availability.
The lesson here was that cloud services were instantly popular with customers even though these early efforts left much room for improvement. That’s exactly what I look for when considering technologies to personally focus upon, and I’ve worked exclusively on cloud services since 2005.
A bit less than a year after that, S3 was announced and the pricing just seemed crazy. I’d worked in database and storage all my life, thought I knew what’s possible, and S3’s pricing simply wasn’t. 15 cents per GB per month was instantly game-changing and there are now over 100 trillion objects stored in S3.
Around that same time, I wrote a good-sized S3 application and tested it extensively prior to demoing to the CEO and CTO of the company where I was working at the time. Over the course of a month of test and development, my AWS bill came in at a nearly invisible $3.08.
Around the time I wrote that first S3 app, EC2 was released into beta. This was back in August 2006. I couldn’t get into the early beta program, but wanted to learn more, so I started attending AWS customer events in the Seattle area. I was amazed both by the number of people in the room at these early AWS events and also by the momentum already building behind EC2. Customers were running their entire businesses upon it. EBS didn’t yet exist and EC2 storage was neither redundant nor fault tolerant. And yet creative customers were innovating with a wide variety of recovery mechanisms to avoid or minimize potential data loss from a disk fault.
Customers were finding the speed of innovation opened up by EC2 so empowering for their businesses that they could live with the early limitations and the lack of durable storage options. Between the low cost of EC2 and the ability to scale at will, huge customer momentum was already building just months after the start of the beta program. Even back then, it was already super clear that hosted services were the future and, if I wanted to be part of the next generation of server-side computing, I needed to be part of AWS.
-
- Interview I did for the EC2 15-year celebration: 15 years of EC2 Innovation
- Short Animated History of EC2: EC2 History
When I joined AWS it was a startup but the paces remains as fast as ever and I’m still learning every day.
Thanks Peter. You asked what I learned about how S3 could possibly have been pricing so differently from the rest of the industry. It was a combination of factors. A custom storage fleet rather than enterprise class file servers. Retail margins rather than enterprise computing margins. A small and very “can do” team that was moving fast. An amazing operational focus where utilization was kept very high. Absolutely no frills anywhere. The data centers weren’t pretty, the offices weren’t pretty, the storage servers were minimalist designs that delivered reliable operation over long lives. They were simple no frills designs. None all that surprising and yet, in aggregate, the price that S3 entered the market at was startlingly different from the rest of the industry.
Hi James hope you are well and congratulations on the expanded role.
Curious if you had time to look into claims by cloudflare about the magnitude of the cost gap between their bandwidth services and those of Amazon. In one blog post they concluded they were the most superior solution Jon bandwidth cost vs aws. This as I am sure you know follows many similar claims by the CEO leading up to today’s announcement of the superiority of their new S3 like service. I realize all competing claims should be viewed with a healthy dose of skepticism but it is rare to see a small company CEO make such forceful claims on infrastructure cost and performance va aws given heritage of innovation under your leadership and that of the infra team at AWS. Any thoughts would be appreciated. Best
Our industry has a large supply of forceful claims and I’ve been known to have a few myself. But, in this forum, I try to stay focused on technology and my personal views on technology– if you want a perspective from Amazon, I would recommend contacting the Analyst Relations team.
James,. Thanks for sharing that. So much of the personal history of tech is lost. I’m curious what you learned when you joined Amazon that explained why your earlier estimates of cost were so far off? I watched the growing power of AWS from Palo Alto where it became clear that more an more startups were using that instead of having to buy lots of computers with quite dramatic impact on the need for VC investment.