Enabling Real-time Streaming: Our Investment in Materialize

By: Logan Bartlett and Sai Senthilkumar

Data is exploding — IDC estimates that over ~140ZB of new data will be created in 2024. The majority of that data is transactional, i.e. when an event or ‘transaction’ occurs like purchasing something online or in a store. Up until a few years ago, data teams have been batch analyzing transactional data — meaning they’re processing a group of transactions after it has been collected over a period of time. Batch processing gave rise to Hadoop and a whole generation of data analysis tools.

What’s been fascinating to observe over the past couple of years is the rapid rise of real-time data, also known as streaming data. In fact, the fastest-growing segment of that 140ZB figure is real-time data, which is predicted to make up 25% of all data generated. As storage becomes cheaper, and computing becomes more powerful, businesses and organizations across a wide range of industries are finding more and more use cases for real-time data as it becomes mission-critical to their operations. Hospitals make use of real-time data to determine the number of open beds available to patients. Exchanges like Nasdaq and Binance need real-time data to fill billions of dollars worth of trades each day. Uber and Lyft need real-time data to identify a driver’s location and ETA for ride-hailing users. Streaming data use cases are legion.

Yet, there is a massive gap in the market for enterprise-grade streaming systems. Existing databases and current streaming solutions force engineers to concede tradeoffs in accuracy, consistency, and latency. They have introduced query languages that are complex, forcing practitioners to learn a new standard. In many instances, real-time data use cases become batch-oriented ones as teams lack the tools to enable live processing of data.

Enterprises have been looking for a comprehensive solution that elegantly supports streaming analysis without requiring restrictive semantics.

Enter Materialize

Materialize is a streaming database for real-time analytics. Its mission is straightforward: to make streaming data operational and simple. Materialize allows teams to query data using industry-standard SQL — no need to learn a new construct. Queries with multi-way joins and complex aggregations arrive within milliseconds, creating a new standard for latency.

Under the hood, Materialize’s engine is built from the ground up on the award-winning Timely Dataflow project, spanning nearly a decade of cutting-edge research led by Materialize Chief Scientist Frank McSherry. Users can reliably query and receive the correct results from underlying data systems even with late-arriving data (a challenging problem known as eventual consistency in the database world). With Materialize, users finally have a robust, no-compromise streaming solution.

In the past six months, Materialize has quickly grown its community to nearly 1,000 developers and landed customers across a broad set of sectors. They continue to introduce innovative new features to the streaming community and we are very excited after the recent launch of Materialize Cloud.

Our Partnership

We’re thrilled to share that we’re leading Materialize’s Series C and joining in this exciting journey to build the future of real-time data.

We have known and been excited about the team for a long time. We first met Arjun Narayan, co-founder and CEO, back when he was a phenomenal engineer at Cockroach Labs, a Redpoint-backed database business. Frank co-invented differential privacy and led the Timely Dataflow project at Microsoft nearly a decade ago, laying the groundwork for Materialize. From the moment we met the team, we were blown away by their technical depth, market acumen, and deep commitment to building products that delight their customers.

At Redpoint, we love working with founders, entrepreneurs, and teams that can connect the dots across their unique backgrounds to imagine something bold and new. Arjun and Frank bring clarity to a complex problem at an opportune time and we are thrilled to be partnering with the Materialize team as they scale this foundational technology and company. We are also excited to join Kleiner Perkins and Lightspeed on this journey.

We at Redpoint have been fortunate to partner with category-defining enterprise data businesses like Snowflake, Looker, and Cockroach Labs as well as infrastructure-focused companies Stripe, Twilio, and HashiCorp and believe Materialize is well on their way in creating an enduring brand. We couldn’t be more excited!