Building Modern ELT Pipelines in Snowflake
How to realize Data Pipelines with SQL and Tasks in Snowflake
Modern Data Warehouses often use a modern ELT approach instead of classical ETL. This article shows you how to realize that certain approach.
In Snowflake, this approach can be implemented relatively well, as already mentioned, e.g. with Tasks and SQL. But first a small recap.
Recap: Data Lakehouse and ELT Approach
Large amounts of data consume a lot of computing power when they are transformed. It is often no longer possible or necessary to distribute the data in the transport, when the data is first stored raw in a Data Lake, for example. From there, they can be transformed for further use cases. For example, Self Service BI, ML, etc. This would be a typical ELT approach.
This approach is called a Data Lakehouse. The Data Lakehouse should combine the advantages of Data Lakes and Data Warehouses into a hybrid concept. Read more about the Data Lakehouse concept here.
