The Next Big Thing

The article below is a restricted version of what I view to be the next big thing. If I was doing a start-up today, it would be data analysis and optimization as a service. The ability to run real time optimization over an understandable programming platform for a multi-thousand node cluster is very valuable to almost every industry. The smaller their margins, the harder it is for them to afford to not to use yield management and optimization systems. The airlines showed us years ago that yield management systems are a big part of what it takes to optimize the profitability of an airline. Walmart did the same thing to brick and mortar retail. Amazon applied the same for online commerce. The financial community has been doing this for years. The online advertising business is driven by data analysis. What’s different as a service is that the technology can be made available to the remaining 99% of the business world. Those that don’t have the scale to afford doing their own.

Storing blobs in the sky is fine but pretty reproducible by any competitor. Storing structured data as well as blobs is considerably more interesting but what has even more lasting business value is the storing data in the cloud AND providing a programming platform for multi-thousand node data analysis. Almost every reasonable business on the planet has a complex set of dimensions that need to be optimized. The example below is a good one but it’s only one. Almost every business can be made more profitable by employing some form of yield management. And, it’s ideally set up for a service based solution: the output data is small, the algorithms to be run are small, the data that forms the input is HUGE but it accumulates slowly over times. Historically large in aggregate but not that large per unit time. The input data per unit time is of manageable size and could be sent to the service. The service provides petabytes of storage and thousands of processors and a platform to run data analysis and optimization programs. This service would also be great for ISVs with companies springing up all over to write interesting algorithms that can be rented and run in the infrastructure.

–jrh


From: Jeff Carnahan
Subject: Fluidity better than perfection?

Good reading for a Saturday morning…

Flight Plan

The math wizards at Dayjet are building a smarter air taxi–and it could change the way you do business.

From: Issue 115 | May 2007 | Page 100 | By: Greg Lindsay | Photographs By: Jill Greenberg and Courtesy DayJet

It’s only fitting that a service pitched to traveling salesmen should find itself confronting an especially nasty version of what’s known as the “traveling-salesman problem.” Stated simply: Given a salesman and a certain number of cities, what’s the shortest possible path he should take before returning home? It’s a classic conundrum of resource allocation that rears its ugly head in industries ranging from logistics (especially trucking) to circuit design to, yes, flesh-and-blood traveling salesmen: How do you minimize the cost and maximize your efficiency of movement?

Solving for X: Once the FAA clears the way for the Eclipse 500, Iacobucci will get to see how good his models really are.

Back in 2002, that was the question facing DayJet, a new air-taxi service hoping to take off this spring. Based in Delray Beach, Florida, DayJet will fly planes, but its business model isn’t built around its growing fleet of spanking-new Eclipse 500 light jets. It’s built on math and silicon, and the near-prophetic powers that have in turn emerged from them. “We’re a software and logistics company that only happens to make money flying planes,” insists Ed Iacobucci, an IBM (NYSE:IBM) veteran and cofounder of Citrix Systems (NASDAQ:CTXS), who started DayJet as his third act.

The advent of affordable air taxis has been heralded by a steady drumbeat of press over the past few years, with an understandable fixation on the sexy new technology that’s generally credited with making the market possible: the planes. The Eclipse 500 is a clean-sheet design for a tiny jet that seats up to six and costs about $1.5 million (the Federal Aviation Administration may clear it for mass production as early as next month). It is also the most fuel-efficient certified jet in the sky. Cessna, meanwhile, has rolled out its own, if pricier, “very light jet” (VLJ), with Honda’s (NYSE:HMC) set to appear in 2010. No less an authority than The Innovator’s Dilemma author and Harvard Business School professor Clayton Christensen has mused in print that the E500 and its ilk “could radically change the airline industry” by disrupting the hub-and-spoke system we all know and despise.

But Iacobucci, who wrote a check long ago for more than 300 orders and options on Eclipse’s first planes, isn’t relying on the aircraft to make or break him. Instead, it’s his company’s software platform–and the novel way it attacks the traveling-salesman problem–that will set DayJet apart. On day one of operations, flying from just five cities in Florida with only 12 planes, DayJet’s dispatchers will already have millions of interlocking flight plans to choose from. As the company’s geographic footprint spreads (with luck) across the Southeast–and as its fleet expands as well–the computational challenge only gets worse. Factor in such variables as pilot availability, plane maintenance schedules, and the downpours that drench the peninsula like clockwork in the summer, and well, you get the idea: Finding the shortest, fastest, and least-expensive combination of routes could take every computer in the universe until the end of time.

“I knew what the complexities were and how the problem degenerates once you reach a threshold,” Iacobucci says. So he didn’t try to find the optimal solution. Instead, DayJet began looking for a family of options that create positive (if imperfect) results–following a discipline known as “complexity science.”

For the past five years, with no planes, pilots, or customers, DayJet has been running every aspect of its business thousands of times a day, every day, in silicon. Feeding in whatever data they could find, Iacobucci and his colleagues were determined to see how the business would actually someday behave. When DayJet finally starts flying, they’ll switch to real-time flight data, using their operating system to shuttle planes back and forth the way computers shuttle around bits and bytes.

Iacobucci is an expert at building operating systems–he did it for decades at IBM and Citrix. Because of that, he has zero interest in the loosey-goosey world of Web 2.0. He sees the next great opportunities in business as a series of operating systems designed to model activities in the real world. DayJet looks to be the first, but he has no doubt there will be others, and that new companies, and even new industries, will appear overnight as computers tease answers out of previously intractable problems.

Which brings us back to the traveling salesmen. Iacobucci says his computer models predict that DayJet’s true competitors are not the airlines, but Bimmers and Benzes–he says 80% of his revenue will come from business travelers who would otherwise drive. In other words, DayJet, which closed an additional $50 million round of financing in March, is creating a market where none exists, an astonishing mathematical feat. To get there, all Iacobucci needed was five years, a professor with a bank of 16 parallel processors, two so-called Ant Farmers, and a pair of “Russian rocket scientists” who, it turns out, are neither Russian nor rocket scientists.

“This is way nastier than any of the other airline-scheduling work we’ve ever done,” says Georgia Tech professor George Nemhauser, whose PhD students have been helping to map the scope of DayJet’s mountain-sized scheduling dilemma. “You can think of this as a traveling-salesman problem with a million cities, and that’s a problem DayJet has to solve every day.”

Tapping into the school’s computing power, Nemhauser and his students have figured out how to calculate a near-perfect solution for 20 planes in a few seconds’ worth of computing time and a solution for 300 planes in 30 hours. But as impressive as that is, in the real world, it’s not nearly enough. That’s because in order for DayJet’s reservations system to succeed, Iacobucci and company need an answer and a price in less than five seconds, the limit for anyone conditioned to Orbitz or Expedia (NASDAQ:EXPE). Because DayJet has no preset schedule–and because overbooking is out of the question (DayJet will fly two pilots and three passengers maximum)–any request to add another customer to a given day’s equation requires its software to crunch the entire thing again.

One of Iacobucci’s oldest pals and investors, former Microsoft (NASDAQ:MSFT) CFO and Nasdaq chairman Mike Brown, pointed him toward a shortcut–a way to cheat on the math. Brown had retired with his stock options to pursue his pet projects in then bleeding-edge topics such as pattern recognition, artificial intelligence, nonlinear optimization, and computational modeling. His dabblings led him first to Wall Street, where he invested in a trading algorithm named FATKAT and eventually to Santa Fe, New Mexico, ground zero for complexity science.

Iacobucci says 80% of his revenues will come from travelers who would otherwise drive. DayJet, in other words, is creating a market where none existed, an astonishing mathematical feat.

Invented by scientists at the nearby Los Alamos National Laboratory in the 1980s, complexity science is a gumbo of insights drawn from fields as diverse as biology, physics, and economics. At its core is the belief that any seemingly complex and utterly random system or phenomenon–from natural selection to the stock market–emerges from the simple behavior of thousands or millions of individuals. Using computer algorithms to stand in for those individual “agents,” scientists discovered they could build fantastically powerful and detailed models of these systems if only they could nail down the right set of rules.

When Brown arrived in town in the late 1990s, many of the scientists-in-residence at the Santa Fe Institute–the serene think tank dedicated to the contemplation of complexity–were rushing to commercialize their favorite research topics. The Prediction Co. was profitably gaming Wall Street by spotting and exploiting small pockets of predictability in capital flows. An outfit called Complexica was working on a simulator that could basically model the entire insurance industry, acting as a giant virtual brain to foresee the implications of any disaster. And the BiosGroup was perfecting agent-based models that today would fall under the heading of “artificial life.”

By the time Iacobucci mentioned his logistical dilemma to Brown in 2002, however, most of Santa Fe’s Info Mesa startups were bobbing in the dotcom wreckage. But Brown knew that Bios had produced astonishingly elegant solutions a few years earlier by creating virtual “ants” that, when turned loose, revealed how a few false assumptions or bottlenecks could throw an entire system out of whack. A model Bios built of Southwest’s cargo operations, for example, cost $60,000 and found a way to save the airline $2 million a year.

Brown proposed that Iacobucci supplement his tool kit with a healthy dose of complexity science. Iacobucci was already hard at work building an “optimizer” program that employed nonlinear algorithms and other mathematical shortcuts to generate scheduling solutions in seconds. But what he really needed, Brown suggested, was an agent-based model (ABM) that would supply phantom traveling salesmen to train the optimizer. Without it, he’d essentially be guessing at the potential number and behavior of his future customers. “Eddy took no convincing,” Brown says. “He was telling me, ‘Get some guys down here and let’s do this.'”

Brown dug up the Ant Farmers, a pair of Bios refugees and expert modelers named Bruce Sawhill and Jim Herriot. Sawhill had been a theoretical physicist at the Santa Fe Institute, while Herriot had been a member of the original team that invented Java at Sun Microsystems (NASDAQ:SUNW). Together, they’re DayJet’s own Mutt and Jeff, with Herriot playing congenial science professor and Sawhill his mischievous sidekick.

Meanwhile, to build the optimizer, Iacobucci recruited his pair of Russian rocket scientists: Alex Khmelnitsky and Eugene Taits, mathematical wizards he’d hired once before at Citrix. Rather than tackle every scheduling contingency via brute-force computing, the not-Russians cheated by slicing and dicing them into more manageable chunks. They used opaque mathematical techniques such as heuristics and algebraic multigrids, which elegantly subdivide a sprawling problem like this one into discrete patches that can be solved (within limits) simultaneously.

Ironically, the more they slaved over the problem, the less it seemed that throwing a perfect bull’s-eye every time was the key to their salvation. The speed of their solutions was proving to be more crucial. If they could provide DayJet with a minute-to-minute snapshot of near- perfect solutions, the system could essentially run the company for them. DayJet would become faster–both in the air and operationally–than any of its competitors could ever hope to be.

With one team working on modeling demand and the other calculating baroque flight plans, Iacobucci and his engineers then concocted a third software system called the Virtual Operation Center. The VOC runs the company in silicon, feeding the phantom customers inside the ABM into the optimizer, which does its best to meet each of their demands with optimal efficiency and maximum gain. Seen on-screen, the VOC is a time-lapse photograph of DayJet’s daily operations, also drawing upon maintenance and real-time weather information to produce a final data feed that factors in nearly every facet of the business. Iacobucci compares each run of the VOC with a game of baseball in which the ABM is continually pitching to the optimizer; DayJet has already played several thousand lifetimes’ worth of seasons.

Armed with its real-time operating system, DayJet is pursuing a very different idea of optimality than, say, the airlines. With their decades of expertise in the dark arts of yield management, the airlines know exactly how to squeeze every last dollar out of their seats, which is indeed pretty optimal. But they also lack an effective plan B–let alone a plan C or D–in the event that the weather intervenes and schedules collapse. In fact, while, say, JetBlue (NASDAQ:JBLU) may now finally have a contingency plan or two, DayJet’s business model is nothing but contingency plans.

Herriot offers another sports metaphor: “Total soccer,” popularized by the Dutch in the 1970s, replaced brute-force attacks to the goal with continuous ball movement. “Moving straight to the goal is an excellent way to score, except for one slight problem–the other team,” Herriot says. “They’re a human version of Murphy’s Law. In total soccer, you continually place the ball in a position with not the straightest but the greatest number of ways to reach the goal, the richest set of pathways.”

“Each individual pathway may have a lower possibility of reaching the goal than a straight shot,” Sawhill chimes in, “but the combinatorial multiplicity overwhelms the other team.” The Dutch discovered that a better strategy was a series of good, seamlessly connected solutions rather than a single brittle one.

“The Dutch won a lot of games that way,” Herriot adds. “It also created a different kind of player, a more agile, intelligent one. In some sense, we’re teaching DayJet how to play total soccer.”

In complexity lingo, a chart of all the pathways those Dutch teams exploited would be called a “fitness landscape,” a sort of topographical map of every theoretical solution in which the best are visualized as peaks and the worst as deep valleys. “We’re dealing with a problem where the problem specification itself is changing as you go along,” Sawhill says. “You no longer want to find the best solution–you want to be living in a space of good solutions, so when the problem changes, you’re still there.” Fluidity is the greater goal than perfection.

To that end, the company has been changing the problem inside its simulators every day for the past four and a half years, looking for those broad mesas of good solutions. And after a million or so spins of the VOC, DayJet has produced a clear vision of the total market and its likely place in it. Iacobucci expects to siphon off somewhere between 1% and 1.5% of all regional business trips within DayJet’s markets by 2008, with “regional trips” defined as being between 100 and 500 miles. In the southeast states the company initially has its eye on, that’s 500,000 to 750,000 trips a year, out of a total of 52 million, more than 80% of which are currently traversed by car. Yes, DayJet’s life-or-death competition is Florida’s SUV dealerships, not the airlines. DayJet may even help the airlines slightly: The model predicts some customers who fly DayJet one way will take a commercial flight back home.

The reams of data produced by the VOC have already coalesced into a thick sheaf of battle plans framing best- to worst-case scenarios. And having run the scenarios so relentlessly for so long, Iacobucci is now utterly sanguine about his prospects. When I ask over dinner for the dozenth time about DayJet’s presumptive break-even number, he flat out admits there isn’t one. “Within the realm of all realistic possibilities–at least 25% of our projected demand to 125% demand–we maintain profitability.” Even at 25%? “Sure,” Iacobucci replies, “it just takes longer, and takes more [airports], and the margin is much lower. But this isn’t going to be what the venture capitalists call the ‘walking dead.’ If it’s a hit, it’s going to be a hit pretty quickly.”

“We’ll see more companies integrate modeling,” says former Microsoft CFO Mike Brown. “This is just like the Internet: One day no one had heard of it, the next day we were all using it.”

I’m not the only one who has trouble wrapping his head around the numbers, or lack thereof. Iacobucci tells the story of one analyst asked to crunch the numbers ahead of an investment. “He asked a direct question: ‘All I want to know is, what formula do I put into this cell to tell me how you come up with a revenue number?'” Iacobucci says. “I told him, ‘There ain’t no formula to put in that cell! It can’t be done! We’ll sit you down with our modelers, who will explain the range of numbers we came up with, but they can’t be encapsulated in a spreadsheet.'” The would-be investors passed.

Not everyone is so put out by the math involved. Esther Dyson, the veteran technologist and venture capitalist, now runs an annual conference called “Flight School,” in which DayJet has played a starring role. “I have no doubt it will work,” she says, referring to the software, “and I have no doubt they will spend time refining it and that there will be glitches here and there. But I do think Ed knows how to design very highly available systems”–a reference to his days building operating systems–“and that’s exactly what they’re doing.”

Mike Brown, who did ante up and today sits on DayJet’s board, is convinced that businesses big and small will increasingly turn to modeling as a way of developing–or troubleshooting–their business plans, mapping out strategies and market expectations that go far, far beyond spreadsheets and PowerPoint (NASDAQ:MSFT) decks. “We’ll see more and more companies integrate modeling into the heart of their business. This is just like the Internet: One day no one had heard of it, the next day we were all using it.”

Since Iacobucci sees himself as being in the operating-systems business, he has no intention of giving that system away. (He learned that lesson the hard way at IBM.) He doesn’t want to build what he calls “horizontal” software that gets shared, e.g., Web 2.0 and Windows, the two great platforms for which every programmer in Silicon Valley seems to be writing widgets these days. Where everyone else in the business sees limitless opportunities in snap-together applications, Iacobucci sees a playing field so flat as to have no barriers to entry at all, and he doesn’t like it.

According to Dyson, DayJet’s competitors have so far pooh-poohed its software, assuming they’ll be able to buy their own off the shelf at some point. Eclipse Aviation’s Vern Raburn hopes Iacobucci might be persuaded to license his tools, because Raburn’s own business model depends upon air taxis’ taking off. Iacobucci says that isn’t going to happen. “There’s a shift away from building another platform toward building highly integrated, vertical, special-purpose, high-performance systems,” he argues. Iacobucci envisions more companies like his own, in which the competitive advantage resides in custom-built, deeply proprietary, real-world operating systems that don’t just streamline accounting, but become the central nervous systems of entirely new, scalable businesses. He’s looking to build barriers to entry out of brainpower–so much of it that rivals can never catch up. (“It’s like in Dr. Strangelove,” Sawhill quips. “‘Our German scientists are better than their German scientists.'”)

Iacobucci points to Google (NASDAQ:GOOG) as an example of what a vertical system can accomplish. While everyone raves about free services on Google, the largely invisible supercomputers in Google’s data centers are themselves invisibly tackling a variation on the traveling-salesman problem: How do you solve millions of searches in parallel at any given second? “When you get into mesh computing,” the name for Google’s technique, “that’s what it’s all about: managing the complexity,” Iacobucci insists.

But no company has ever built a business model around complexity from the ground up–until DayJet. Thumbing his nose at the prevailing ethos in software circles of “the wisdom of crowds,” let alone that “IT doesn’t matter,” Iacobucci has set out to first invent and then dominate a market he might have otherwise just sold software to. “When we built generic software at IBM and Citrix, the other side would always reverse-engineer it,” he says. “The only thing the customer sees here is an incredible service. This is ‘software as a service.'”

Source: FastCompany.com

James Hamilton, Windows Live Platform Services
Bldg RedW-D/2072, One Microsoft Way, Redmond, Washington, 98052
W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 |
JamesRH@microsoft.com

H:mvdirona.com | W:research.microsoft.com/~jamesrh

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.