Data Lake

Эта последняя статья из серии с обзором книги Learning DDD, в которой мы рассмотрим data mesh. Рекомендую перед чтением этой статьи посмотреть предыдущие три части: 1, 2 и 3. Интересно, что про концепции DWH и data mesh я год назад сам рассказывал в лекции про данные из курса Essential Architecture…

This month, KYVE entered into the Polkadot ecosystem, launching integrations with both Polkadot and Kusama. Little do some people know, this is a major step for our team, coming full circle with our origin story. In early 2021, KYVE’s founders Fabian Riewe and John Letey came together to participate in…

Data Lakehouse is a new technology that can be seen as the superset of both Data Lake and Data Warehouse. Lakehouse covers all analytics workload (BI, AI, ML) on any layout of data (structured or unstructured) and any arrival velocity (high speed like streaming, or slow speed like batches). Data…

with Databricks — I recently joined Databricks, a company founded by the creators of Apache Spark. At the time that I am joining the company they have moved far beyond just “Spark in the Cloud” and I believe they are leading the next revolution in Data Warehousing and Analytics. This blog article starts…

Intro Many of you who have seen my LinkedIn profile must be wondering what my profile headline means. For the longest time, a Data Warehouse has been the gold standard for integrating data from disparate systems into a single database to analyze and derive business insights. After having worked in the…

Background Spending on cloud infrastructure and services is accelerating. According to a recent report by IDC, worldwide “whole cloud” revenues totaled $706.6 billion in 2021, and are forecasted to reach more than $1.3 trillion by 2025. Data from Synergy Research Group confirms this trend, showing growth in on-premises data center spending…

Intro In our previous blog post, part 3 of this series, we walked through the processing of data available from the AWS sample Tickit data. We then used EMR to process it from RAW zone to Trusted with complete integration of AWS Glue (Data Catalog, Schema Registry). …

In this article, I hope to paint a picture of the modern data world and when done you should have a decent understanding of the roles of Data Engineers, Data Analysts and Data Scientists and the technologies that empower them. The Transactional World (OLTP) Imagine we are creating the newest social media startup. Our…