I helped kick off CIDR2024 yesterday with the keynote, Constraint Driven Innovation.  My core thesis is that constraints force innovation.  For example, it was slow hard disks that drove the invention of Write Ahead Logging. But constraints also block innovation. In memory databases first described in the 80s remained largely irrelevant for decades waiting for cost effective very large memory servers to make them practical. In this talk I focus mostly on the latter.  Constraints that have come down or are coming down opening up new opportunities for new database technology improvements.

Believing that H/W acceleration is becoming increasingly practical, we look at what’s happening in the machine learning world and argue that the database market is large enough to justify custom hardware investments.

For those of you that haven’t attended the Conference on Innovative Data Research, have a look at the CIDR2024 program and consider attending the next CIDR.  20 years after attending the first, it remains one of my favorites.

Talk Slides: Constraint Driven Innovation