How My Snowflake Lead Distro Test Turned Out to be Reverse ETL

A year ago, I worked on a small project to help improve our funnel conversion with data-driven lead distribution for Salesforce.com. I referred to the initiative as “Snowflake sales metric sync” to drive awareness to business stakeholders of the utility we were getting with Snowflake. It turns out, we were doing “Reverse ETL”. This article shares some of the lessons learned along the way and some thoughts about where reverse ETL is headed.

My Use Case and Results

A rep with a fully loaded pipeline working to close deals at the end of the month, even as a high performer, naturally has less time to work brand new leads. My hypothesis was sales reps with more capacity at end of month would produce better results knowing speed to lead is key to conversion. Using a capacity metric and automated load balancing, we would give leads to reps with more capacity and then evaluate the performance of end of month leads.

Low Level of Effort Solution

All of the data and metrics I needed was already available in Snowflake for reporting, so the process to push those measurements back into a Salesforce object using Azure Data Factory was quite simple.

The transformation work was prepared using Datameer on top of Snowflake which I had previously written about on Medium: Slice Through your Snowflake Data like a Buzzsaw with Datameer

If you were wondering how the load balancing and lead distro magic happens, it’s all performed thanks to Distribution Engine for Salesforce.

Turns out I was doing reverse ETL!

I admit, I had never really given thought to the term “reverse ETL” until I went to the Snowflake conference in 2022. I have stressed for years that getting hung up on technology trends without clear business outcomes is just noise. My recommendation is always to align and name projects and initiatives to results and not technology buzzwords.

High Hopes for Outcomes with Inconclusive Results

I would love to have had a windmill slam dunk of a success story, but the reality was my test was flawed because I could not properly split my population in a meaningful way. The gains I measured were marginal at best. This is a reality for smaller organizations working toward data driven processes where time and scale are not on your side. However, the method and lessons learned were extremely useful. More import, the process to make 20+ sales reps metrics available in Salesforce opened the doors for more tests!

What Exactly is Reverse ETL?

For those of you still wondering what Reverse ETL is… First, ETL stands for Extract, Transform, Load. It is a process used in data integration where data is extracted from multiple sources, transformed to fit a certain format or structure, and then loaded into a central database for analysis and reporting. It is synonymous with how traditional enterprise data warehouses are populated with data which requires careful structuring and design considerations for data structure, scalability, and governance.

Reverse ETL is a newer concept that involves the opposite process. Instead of bringing data into a central data store, Reverse ETL takes data from a central data store and distributes it to various applications and systems where it is needed. This approach allows for more flexibility in how the data is used and can help organizations to quickly and easily get data to where it’s needed without having to go through a time-consuming data transformation process upfront.

Reverse ETL is useful in modern data warehousing and data integration setups, partially because the barrier to do so is so low. At the end of the day my opinion is Reverse ETL is just a buzzword that may or may not ever reach massive adoption.

What are Common Use Cases for Implementing Reverse ETL

Data Driven Processes is the use cases that directly impact your KPI is the best use case for reverse ETL. When you pull disparate data back into your business application in the form of actionable information, as long as you can impact your metrics or address a pain point at scale, you are on the right track. The context here for “data-driven” is mostly removing human decision making.

Reporting is another very popular use case that I see other folks gravitate to. I would argue that the reporting driven use cases for reverse ETL better have a have high impact above and beyond your standard BI and analytics solution. That said, with the emergence of AI, having more data on hand within your business applications will be increasingly important faster than you think.

Data migration is a clear Reverse ETL use case where using your cloud data lake / warehouse to stage analyze and load data into your business application makes sense. Typically, this occurs when you are expanding or consolidating business applications.

My Thoughts on Future of Reverse ETL

The real innovation I am seeing in the world of data is the speed and ease for preparing and delivering meaningful, useful data, insights, and scores back into the business process seamlessly. With speed comes the word of caution that all paths always lead back to data governance!

Generative AI is something you should consider for any future planning. Salesforce.com for example, has already integrated ChatGPT into numerous business workflows. AI bots will not go hunting through your data warehouse and BI tools for facts and figures…yet. Reverse ETL those facts and figures into your business applications where business users and OpenAI reside. Nothing makes humans sound more credible than having statistical facts on hand. Arm OpenAI with those same facts, and we should see some incredible results. I am employing this for my own onboarding and activation communications and look forward to writing about it soon.

Commoditization of Reverse ETL data movement could occur as more customers gravitate to cloud native data platforms like Snowflake, Google Cloud, and others. I have already experienced this with this with Salesforce connector for Snowflake which worked beautifully, but was a bit pricey. For now, the third party ecosystem does it better and cheaper.

Conclusion

While my focus and energy had traditionally been Business Intelligence and reporting, leveraging reverse ETL and newer machine learning platforms to push analytics back into Salesforce.com has been an eye opening experience finding new interesting approaches to solving age-old problems.