Amazon doesn’t release much about its inner workings which is unfortunate in that hey do some things notably well and often don’t get credit. My view is that making some of these techniques more public would be a great recruiting tool for Amazon but I understand the argument for secrecy as well.

Ronny Kohavi recently pointed me to this presentation on A/B testing at Amazon. It’s three years old but still well worth reading. Key points from my perspective:

· Amazon is completely committed to A/B testing. In past Bezos presentations he’s described Amazon as a “data driven company”. One of the key advantages of a service is you get to see in real time how well it’s working. Any service that doesn’t take advantage of this and is just making standard best guess or informed expert decisions, is missing a huge opportunity and hurting their business. The combination of A/B testing and cycling through ideas quickly does two wonderful things: 1) it makes your service better FAST, and 2) it takes the politics and influence out of new ideas. Ideas that work win and, those that don’t show results, don’t get used whether proposed by a VP or the most junior web designers. It’s better for the service and for everyone on the team.

· The infrastructure focus at Amazon. Bezos gets criticized by Wall Street analyst for over investing in infrastructure but the infrastructure investment gives them efficiency and gives them pricing power which is one of their biggest assets. The infrastructure investment also allows them to host third parties which gives Amazon more scale in a business where scale REALLY matters and it gives customers broader selection which tends to attract more customers. Most important, it gives Amazon, the data driven company, more data and this data allows them to improve their service rapidly and give customers a better experience: “customers who bought X…”

· Negative cost of capital: Slide 8 documents how they get a product on day 0, sell it on day 20, get paid on day 23, and pay the supplier on day 44.

· Slide 7 shows what can be done with a great infrastructure investment: respectable margins and very high inventory turn rates.

The presentation is posted: http://ai.stanford.edu/~ronnyk/emetricsAmazon.pdf.

–jrh

James Hamilton, Windows Live Platform Services
Bldg RedW-D/2072, One Microsoft Way, Redmond, Washington, 98052
W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 | JamesRH@microsoft.com

H:mvdirona.com | W:research.microsoft.com/~jamesrh