Data infrastructure has been consistently disrupted by companies that innovate on performance, platform (e.g. cloud), or both. Interfaces are usually an afterthought — perhaps a nice dashboard, installer, or query explorer to accompany the core product.
But machine learning is rapidly transforming enterprise data processing, and traditional interfaces do not translate to these new workloads. In the ML era, innovation in data infrastructure will be led by interface design and frontend engineering.
Data processing workflows have stayed fairly constant in the past. While there have been several hardware/platform shifts that enable faster/cheaper/more storage and computation, the phases are still roughly:
Enterprise software is a pillar of Silicon Valley. However, it is no secret that cloud providers and open-source software have commoditized use cases that were traditionally ripe for enterprise software companies.
This post is geared towards founders, candidates, and investors trying to reason about viable enterprise software ideas. In this brave new world, where do the opportunities lie? It’s actually quite simple — just look at where both cloud providers and open-source are weak.
Where do cloud providers innovate? They take established technology that still needs sales engineers (object storage, VMs, data warehouses) and make it self-serve.
Before AWS, it…
This is for those of us who walk around the world looking for inspiration. We often find it in music and in people, but until today I’ve approached these topics separately.
I usually walk around with headphones on, focused on music, ignoring my surroundings except for the bare minimum — obstacles and crosswalks. Every once in a while I take them off and try to examine the world around me, a very visual experience where I think about each and every thing that I see on the street.
Today I realized you can actually combine these two techniques. Start listening…
Today is my 5 year anniversary at MemSQL. When I tell people I dropped out of school and have been at MemSQL for about 5 years, I get everything from skepticism to shock. Most folks I talk to are planning to stay at a company for 18 months to 2 years.
To me it’s a no-brainer. If you join a healthy startup, you should plan to be there for at least 5 years. Here are the top 3 reasons why:
The more you know about the nature of the…
The terms rowstore and columnstore have become household names for database users. The general consensus is that rowstores are superior for online transaction processing (OLTP) workloads and columnstores are superior for online analytical processing (OLAP) workloads. This is close but not quite right — we’ll dig into why in this article and provide a more fundamental way to reason about when to use each type of storage.
Learner, Former VP of Engineering at MemSQL