Data Platforms

Introduction I recently had a fascinating discussion with Ankit Pareek, a brilliant mind, who posed a challenging question about my recent blog on resiliency. His query focused on understanding the difference between retry and auto heal mechanisms. …

Businesses continue transforming their operations to increase productivity and deliver memorable consumer experiences. This digital transition accelerates timeframes for interactions, transactions, and decisions. Additionally, it generates reams of data with brand-new insights into operations, clients, and competition. Machine learning helps companies in harnessing this data to gain a competitive advantage…

Platys is a tool created by Guido Schmutz, architect and data specialist at Accenture and frequent teacher at various universities. In order to be able to quickly create an environment to investigate a tool, build a proof of concept with a specific combination of technologies or provide all his students…

In a previous article titled “Enhancing Reliability in Data Platform Ecosystems: Key Factors and Best Practices” (refer to: link), I discussed the importance of improving reliability in data platform ecosystems. In this blog post, we will take a closer look at this topic by diving into a specific example. In…

Introduction Building a highly reliable data platform ecosystem is crucial for organizations dealing with large-scale data processing. To achieve this, it is imperative to consider both functional and non-functional aspects of the system. In this blog post, we will explore three key factors — retry, auto-heal, and auto-scale — that…

Introduction Data monetization has become a crucial aspect of business success in today’s digital landscape. With various avenues available, such as APIs and data marketplaces provided by cloud data providers like AWS, Google, Snowflake, and DataBricks, organizations are exploring ways to leverage their data assets. To enable seamless data distribution across…

Exploring the what, why, and how of design docs for data components — and why they matter. — Over the past few years, adopting software engineering best practices has become a common theme within the data engineering space. From dbt’s software-engineering-inspired capabilities to the rise of data observability, data engineers are getting increasingly accustomed to the software engineer’s toolset and principles.

Disclaimer: please note that my opinions or my reflections do not necessarily reflect the beliefs of my employer. — Very recently, I stumbled upon this cartoon and I had to chuckle a bit — but the laughter was quickly replaced by sober realization. Generative AI holds immense potential for organizations across industries and this wave will either bring new revenue streams or unmatched productivity levels to customers who wields…