Design Patterns for Applications Leveraging the Snowflake Architecture
Designing the Key that Opens the Data Treasure Chest
When we talk about data strategies, there are so many treasure hunt analogies, they have become part of our regular vocabulary. We speak of data as being the ‘key’ to becoming successful. The potential that data has, first needs to be ‘unlocked’. We even call an entire modeling approach a data ‘vault’.
But luckily, it becomes easier and easier to ‘unlock’ the potential of data. At Snowflake, we see more and more customers and partners that are actively monetizing these ‘keys’ in the form of data-driven applications. Because why go on a treasure hunt when everything is already right there?
Snowflake is a powerful platform to support these data-driven applications. However, a lot of application builders struggle with making the right architectural decisions:
- Where and how would you use Snowflake?
- Who would own the application vs. the data?
- And what model suits the situation best?
In this blog post I will outline and compare the different deployment models that we see. (Scroll down to the end to find a comparison table.)
Why build applications on top of Snowflake?
So why would you build an application using Snowflake? Let’s start by looking at the typical application (see image below). Performance is often a challenge when it comes to getting and analyzing the data, but this is exactly what Snowflake was built for. Snowflake’s unique architecture is the main reason why many customers and partners use Snowflake to power this application workload.
- Performance — With Snowflake you can easily scale your compute clusters to match the the power needed for individual workloads (vertical scaling), but what really makes the difference when it comes to applications is the ability to scale out automatically in case of concurrency (horizontal scaling): Whenever there are many requests at the same time -something that can be quite unpredictable when it comes to applications (i.e. multiple users on the same app at the same time)- Snowflake will automatically spin up extra clusters to deal with the load.
- No over-provisioning: The flexibility in scaling your compute clusters also has a positive effect on costs. Scaling horizontally means that your compute clusters are only active when actually needed and you are not over-provisioning at the more ‘quiet’ moments.
- Low Maintenance — Snowflake is operated as a SaaS, so you as application provider do not have to manage the underlying database yourself. You can easily plug into the capabilities of Snowflake as a data platform.
This model is not enough anymore…
Even though this model looks beneficial compared to working in a traditional way, many challenges have not been addressed:
- This is not solving the problem the many data leaders are trying to deal with: Getting rid of data silos and creating a centralised Single Source of Truth;
- Many SaaS providers are still locking access to data, making it hard to get data out;
- There are no new processes or infrastructures available to access the data, making it cumbersome and slow (and often costly) to get the data ‘out’ when it’s needed.
All in all, we need to think about different models in order to remove the friction for the customer to leverage data created by SaaS applications.
Managed Application
The first deployment model that solves the above challenges is what we call the Managed Application. In this case the app provider stores and processes customer data in its own data platform (eg. traditional SaaS model) and in addition you can choose to differentiate your product or monetize your data sharing it with your customers.
You (application builder) are the SaaS provider, you build and own the application code and logic. You also own the Snowflake Account in order to store all the data you need for your application to run from an analytics perspective (see image below). Thanks to Snowflake’s Data Sharing capabilities you are able to deliver live, ready-to-query data to the app consumers, without having to go through ETL/ELT processes.
The customer is able to see only the data that you allow them to see, so this also means you, as app provider, are responsible for the data governance.
Benefits
With this model, you offer your app consumers a fully managed service, and this comes with some benefits as well:
- High ‘Total Addressable Market’ — Since you are providing a fully managed service, there are hardly any prerequisites before you can spin up the app. You can sell to almost anyone, whether the customer is also a Snowflake User or not (using Reader Accounts), and whether it is B2B or B2C.
- Economy of Scale — This deployment model is also the easiest to use when you are going after an Economy of Scale model and is therefore an interesting design model from a business point of view.
Connected Application
The second deployment model is the Connected Application. In this case the SaaS-provider owns only the code and the business logic and the app stores and processes data that is owned by the customer with the customer’s own Snowflake Account (see image below). The difference with the Managed Application is that you do not own and process the data in the Snowflake account directly: You are plugging the code into the customer’s Snowflake account.
Benefits
Connected applications give customers control over their own data, allowing them to maintain a single source of truth and centralized governance. This deployment model removes the friction of having customer data in your environment as app provider. Because the customer continues to own their data and the compute part of Snowflake, they can make their own decisions and govern it in the way they want, which comes with benefits for both sides:
- Accelerated sales process — This architecture eliminates the need for the SaaS-provider to put in place compliance, and governance mechanisms on the data, because the data stays where it is (with the customer). This can significantly speed up the selling process of such an app, because there is less legal friction and less worry about a potential lock-in.
- Address specific markets — Because the customer still owns the data, this also means you can address specific markets where data privacy regulation is more important (like government or financial services).
- Lower costs / higher margin — Because the customer owns the storage and the compute side in Snowflake, it also means that this part of the costs lie with them and not with you as app provider. This model gives you a lot more freedom as to the business model you want to implement.
Native Application
Last but not least, you can soon also build Native Apps with Snowflake. Building Native Apps with Snowflake is currently in Private Preview and more is to come later (if you want to receive the latest updates, you can sign up here). A Native App is not competing with a Managed App or a Connected App, it is just a different way to monetize business logic that you can build and store in Snowflake. Think of a Native App as a way to bundle Snowflake functionality (stored procedures, user defined functions, or other types of business logic), which you can then distribute and monetize through the Snowflake Marketplace -which is not just a data marketplace anymore.
Snowflake Users can look for pieces of logic and entire applications, and buy and consume directly in their account as a customer. This is then stored on the customer side and there is no direct link anymore with you as a provider.
Benefits
Building a Native Application with Snowflake would be mostly a strategic choice. Distributing through the Snowflake Marketplace and basically ‘handing over’ your code, data, and logic, comes with quite some benefits. Building Native Apps with Snowflake is still in Private Preview, but you can already have a look at this blog post to dive into the benefits of Native Apps for both providers and customers.
What is better?
So what is better? As with many questions, the answer is: It depends. To complicate things even more, the reality is that most of the time, we see both the Managed and Connected deployment model for the same application. If you already have an application, the easiest way is to ‘unplug’ your existing system and plug it into Snowflake (Managed App). Later on you could move to a Connected App to address more specific (regulated) markets. When starting from scratch (e.g. start-ups), we often see a start with the Connected App, with a transition to the Managed App later on to address a larger market. If you want to dive deeper into this comparison, have a look at this blog post on the differences between de Managed and Connected App and the benefits.
The Native App is an entirely different way of monetizing applications, logic, and data through the Snowflake Marketplace.
Find below a table with a comparison on the most important considerations:
Summary
We speak of data -when used right- as the key to unlock the potential it has. But why reinvent a key when it is already available? App builders (both Snowflake customers and partners) can easily leverage Snowflake to power their data-driven applications and monetize this ‘key’. Even though the more typical way of leveraging Snowflake to power an app already comes with great benefits, you can implement three deployment models that deal with the challenges that are still there. The biggest differences are in who owns the data, and who owns the code. There is no good or bad deployment model and often, they go hand in hand in a hybrid form. Both architectural and go-to-market decisions have to be made to decide on the best model.
