One of the most important attributes needed in a cloud solution is what I call cloud data freedom. Having the ability to move data out of the cloud quickly, efficiently, cheaply, and without restriction is a mandatory prerequisite in my opinion to trusting a cloud. In fact, you need the ability to move the data both ways. Moving in cheaply, efficiently, and quickly is often required just to get the work done. And the ability to move out cheaply, efficiently, quickly, and without restriction is the only way to avoid lock-in. Data movement freedom is the most important attribute of an open cloud and a required prerequisite to avoiding provider lock in.
The issue came up in the comments on this post: Netflix on AWS where Jan Miczaika asked:
James, as long as Amazon is growing constantly and has a great culture of smart people it will work out fine. Should the going ever get tough (what I of course don’t hope) these principles may be thrown overboard. It would not be the first time companies sacrifice long-term values for short-term profits.
This is all very hypothetical. Still, for strategic long-term planning, I believe it should be taken into account.
And I responded:
Jan, it is inarguably true that there have been instance of previously good companies making incredibly short sighted decisions. It has happened before and it could happen again.
The point I’m making is not that there exists any company that is incapable of damn dumb decisions. My point is that the cloud computing model is a huge win economically. I agree with you that no company is assured to be great, customer focused, and thinking clearly forever. Disasters can happen. That’s why I would never do business with a cloud provider that didn’t have great support for export of LARGE amounts of data cost effectively. Its super important that the data not be locked in. I don’t care so much about the low level control plane programming model — I can change how I call those APIs. But its super important that the data can be moved to another service easily. And, this export service has to be cheap and there is no way I would use the network for very high scale data movements. You have to assume that the data is going to keep growing and so its physical media export that you want. Recall Andrew Tenenbaum’s “Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway” (sneakernet).
I’m saying you need to use cloud computing but I’m not saying you should trust one company to be the right answer for ever. Don’t step in without a good quality export service based upon physical media at a known and reasonably price.
This morning AWS Import/Export announced that the service is now out of beta and it now supports a programmatic, web services interface. From the announce letter of earlier today:
AWS Import/Export accelerates moving large amounts of data into and out of AWS using portable storage devices for transport. The service is exiting beta and is now generally available. Also, a new web service interface augments the email-based interface that was available during the service’s beta. Once a storage device is loaded with data for import or formatted for an export, the new web service interface makes it easy to initiate shipment to AWS in minutes, or to check import or export status in real-time.
You can use AWS Import/Export for:
· Data Migration – If you have data you need to upload into the AWS cloud for the first time, AWS Import/Export is often much faster than transferring that data via the Internet.
· Content Distribution Send data you are computing or storing on AWS to your customers on portable storage devices.
· Direct Data Interchange – If you regularly receive content on portable storage devices from your business associates, you can have the data sent directly to AWS for import into your Amazon S3 buckets.
· Offsite Backup – Send full or incremental backups to Amazon S3 for reliable and redundant offsite storage.
· Disaster Recovery – In the event that you need to quickly retrieve a large backup stored in Amazon S3, use AWS Import/Export to transfer the data to a portable storage device and deliver it to your site.
To use AWS Import/Export, you just prepare a portable storage device, and submit a Create Job request with the open source AWS Import/Export command line application, a third party tool, or by programming directly against the web service interface. AWS Import/Export will return a unique identifier for the job, a digital signature for authenticating your device, and an AWS address to which you ship your storage device. After copying the digital signature to your device, ship it along with its interface connectors and power supply to AWS.
You can learn more about AWS Import/Export and get started using the web service at aws.amazon.com/importexport.
Also announced this morning: AWS Management Console for S3 and 3 days ago: Amazon Cloudfront adds HTTPS Support, Lower prices, and Opens an NYC Edge Location. Things are moving pretty quickly right now.
b: http://blog.mvdirona.com / http://perspectives.mvdirona.com