The Madman and the Data Warehouse in a Box on Lakehouse
We were sitting at lunch, Sweetwater BBQ had an amazing sweet smokey flavor that hung around all afternoon. The times were good. Quite often when going out to lunch with co-workers, before the Pandemic changed our lives, when working at the office was common, we would get into deep discussions about grand schemes to make it big. This one crazy idea stuck with me for a very long time. This was 2018, and back then the whole ‘cloud’ thing was early days, and Snowflake was struggling with going public as an expensive premium cloud data warehouse. My colleague and I were discussing how the cloud could revolutionize our industry. I had my doubts that enterprises would ever adopt the cloud because of security, and they wouldn’t directly own and control it. He explained that most Enterprise data are in co-located data centers where they have to pay more money to third-party companies to manage the data center, and still have to maintain all the systems with their staff and pay more for vendor software, which you have to maintain as well. As soon as these huge companies like Amazon with AWS, Microsoft with Azure, and Google with GCP figure out scale, no one is going to buy from a stitched-together co-located data center with high administration costs, and software management costs. You see these clouds built capabilities that plugged directly into their software, that they sold as a…
