The top four things to get right when implementing Snowflake
Snowflake is recognised by many as the best cloud data platform. However, as with any cloud data platform, the right implementation is key.
- What steps should you take to implement Snowflake successfully and what should you watch out for?
- How do you make the transition from your current on-premise data warehouse to Snowflake on-time and on-budget?
- What are the four key factors you must consider to ensure implementations run smoothly?
In this post, we’ll guide you through the top four things which are essential to get right at the implementation stage.
This article is the first post in a five-part Snowflake blog series.
1. Snowflake Deployment Type
There are three deployment options in Snowflake and choosing the right option for your organisation determines whether your implementation will be a success or not in the long run.
The three deployment options are:
- Three separate accounts. One for each environment — DEV, TEST and PROD
- Two accounts. One for production and the other for both DEV and TEST
- A single account for DEV, TEST and PROD environments
The recommended option is the Single account because it makes data management and administration easy.
2. Architecture
Designing the architecture in the right way is critical because, without the correct architecture (or foundations), the structure built on it will be shaky and very hard to manage.
Below is the Snowflake reference architecture:
The reference architecture can be built upon to fit in with your overall enterprise data architecture. Data lakes can be built on top of the file landing zone or integrated into Snowflake within the staging area.
3. Data Modelling
Building a data warehouse necessitates a modelling approach tailored to your organisation. Snowflake supports the three main data warehouse modelling approaches.
Deciding upfront which approach to follow is very important and will save you time going forward:
- Inmon Approach — using data marts as physical separation from the enterprise data warehouse. Very good for strategic enterprise-wide integration but with high start-up cost.
- Kimball Approach — using the dimensional model (star schemas) to organise data in a dimensional data warehouse. Ideal for tactical individual business requirements. The start-up cost is low.
- Data Vault Approach — a hybrid of Third Normal Form (3NF) and star schema. This is a relatively new approach compared to Inmon and Kimball but very good for both audit and historical tracking purposes.
We recommend the Kimball approach if you are starting from scratch due to its simplicity.
4. Data Processing
Snowflake supports different data ingestion and integration patterns — scheduled (batch), event-based (near real-time) and streaming (real-time). Evaluating your data use cases will help identify the best pattern to use. You can also pick and mix these patterns to reflect how data flows through your organisation.
The pattern selected will guide the toolset required for data ingestion and integration. There is great support for third-party ETL tools like Ab Initio, Informatica, Talend etc as well as ELT tools such as Matillion and DBT. Hand coding is also supported by Snowflake using programming languages like Python, Go, .NET, Java and Node.js etc.
Below are the recommended approaches to data ingestion and transformation:
Data Ingestion into Snowflake:
Data Transformation within Snowflake:
See also our Four keys to success with Snowflake blog for more information.
How Slalom can help
Slalom has the experience to help your business make the most of Snowflake and set the stage for long-term growth and sustainability.
Our Snowflake credentials include:
- Snowflake Partner of the Year for three consecutive years — 2018, 2019 and 2020
- Over 200 Snowflake certified consultants
- Over 300 Snowflake projects delivered
In our next blog, we explore the different Snowflake deployment options.
David Oyegoke is a Data & Analytics Consultant based in Slalom’s London, UK office. Ashish Billore is a Data & Analytics Solution Architect, also based in Slalom’s London, UK office.
Slalom is a modern consulting firm focused on strategy, technology, and business transformation.
