Speaking Data with Snowflake and Looker
There are probably hundreds of start-ups out there that claim to speak data and most likely each and every one of them takes a different view or approach on this topic depending on the area of business or company culture.
In this story I would like to share our approach on “speaking data” as delivery platform and how we use Snowflake as data warehouse solution together with Looker as a BI platform to achieve that.
What Makes us Speaking Data
Just claiming to speak data and emphasizing the importance of data driven decisions is -if at all- just half the battle. We as the BI team of Seven Senders strive for making the data available for everyone in the company without the need of contacting us. We serve the other departments by providing the infrastructure and knowledge for a decentralized business intelligence architecture and as a data governance instance to make sure that we all have the same understanding of the data provided.
We do so by using Looker as a BI tool on top of our Snowflake data warehouse. This setup allows us not only to model the data on the fly in an flexible ELT process but also to share the same results internally with other departments and externally with customers and partners.
A Data Warehouse as Enabler for Speaking Data
The core of our business model is to connect merchants with prime last mile carriers worldwide. In terms of data this means many different data sources, different types of data and different aggregations. Now imagine to have all those data of different types in various unconnected data silos rotting over time without adding any value to the whole supply chain. This would definitely not be a good starting point for speaking data and for making data driven decisions.
That is why we decided to implement a data warehouse. By bringing together all the data in one central place we had the data where we needed it to be in order to generate holistic data insights.
Why Snowflake as Data Warehouse
Our decision on using Snowflake end of 2017 was mainly due to the following three factors:
- Performance
As stated above we use Looker and thus also our data warehouse both internally and externally. For us it was very important to not make an any compromises in terms on performance towards our customers despite using an ELT approach. Snowflake and its adaptive architecture enable very fast query times needed in such a setup. - Flexible Scale
Different consumers of data have different needs in terms of computing and storage. With Snowflake we are able to adapt the power of our data warehouse depending on the user by using multiple virtual data warehouses with a different scale depending on the user. Aligning the different demands for compute and storage to different supplies of compute and storage also helps to generate saving potential. - Usability
Snowflake is a ready-to-use data warehouse allowing us as a BI team to instantly scale, create, modify warehouses in a user friendly web interface. There is no need for a DevOps even if you are completely without IT background (like myself). Simply login, select the desired warehouse and adjust the computing power by means of a click.
I will share some more detailed insights on our decision for Looker as BI platform on top of Snowflake in an upcoming post.
The Result
By implementing our tool stack with Snowflake and Looker we were instantly able to generate much more value out of our data silos. By bringing the different sources together in one place we expanded our data insights to make data driven decisions. From monthly reporting to daily cost controlling and monthly performance audits. What has not been done before or just irregular in extensive excel work is now possible with a click.
In addition we were able to decrease the time to market for new data insights for our customers in our Sendwise Delivery Software. With our ELT approach enabled by Snowflake we can model and share new data insights in hours to days rather than weeks to months. This of course also helps to widely expand our customer base and with the new setup we do not have to worry about scalability or performance issues.
Take a look at the following video by Snowflake where I summarized our journey (German only, sorry).
