Data Infrastructure

By Shimul Bhowmik and Karthik Golla Pocket Gems Data Infrastructure Team During the pandemic, when our data infrastructure (DI) team was working in different time zones, it was difficult to effectively utilize the engineering resources during on-call. Pre-pandemic, people used to report issues via email, Slack message, or in person…

Improving your data infrastructure and data culture takes more than a flashy pipeline — History likes to repeat itself. To put it another way: we humans have a talent for creating the same problems for ourselves over and over, disguised as something new. Data stacks and data pipelines are no exception to this rule. Today’s data infrastructure landscape is dominated by the “modern data…

Decentralized storage has emerged as a solution to the challenge of providing a highly performant, secure, private, and economical cloud storage solution. Decentralized storage is in the best position to achieve cloud storage goals for end users as the architecture has a more natural alignment to the decentralized architecture of…

Netflix is unquestionably the largest video provider in the world, delivering the most streams to the most customers from the largest video library that is by some estimates almost four times bigger than its closest competitor. No one knows for sure, but that probably translates into Netflix having the largest…

In both my previous and my current job I see that many engineers want to create their own data platform. That by itself is neither good nor bad, I do feel it is often attempted for the wrong reasons or certain critical considerations are overlooked. Over the past 6 years…

The new category of cloud database services emerging — — Originally posted on Techcrunch — One of the most profound and maybe non-obvious shifts driving this is the emergence of the cloud database. Services such as Amazon S3, Google BigQuery, Snowflake, Databricks, and others, have solved computing on large volumes of data, and have made it easy to store…

Excited to share the backstory of our seed investment into Meltano Data is transforming the Enterprise The amount of data being generated and stored has exploded in scale and volume. Every aspect of the enterprise is being instrumented for data with the goal of unlocking new capabilities and building an edge. …

A lot has changed at Cedar since I joined at the start of 2020. We are a multi-product company, have spread to all corners of the US, and can no longer seat most of our New York office at two lunch tables. As a data scientist on the Data Infrastructure…

Today, the team at Data Minded is proud to announce the launch of Conveyor, our managed, cloud-based data platform. Yet another data platform on the market? Well, yes… Yet we believe this one is different. Conveyor takes your data projects from ideation to production. Unlike many other platforms available out…