An Alternative Strategy for Solving ETL (Extract, Transform, Load) for Your Product

Solve data onboarding and enable customers to adopt faster.

Hassan Syyid
Sep 26, 2019 · 5 min read
Source: Pexels
  • For most SaaS B2B firms, the biggest barrier to adoption is getting customer data into their product.
  • Traditional ETL (extract, transform, load) solutions target large enterprise firms. Startups cannot afford to use these, and are reduced to building their own solutions or painstakingly transforming data by hand.
  • Embedded ETL is a newer approach designed to give startups a real ETL tool. Instead of shirking the responsibility of on-boarding customers, providing ETL from within the product translates to vastly reducing the barriers to adoption for your product.

Data is Your Traction Problem

Most B2B software vendors fail to realize that one of their biggest adoption problems is that the data they need to process is not going to come directly into their systems through the beautiful UI that they built.

Quite the contrary — most solutions require data to be transitioned, integrated and copied from other upstream and downstream systems to make the product operational for a customer.

This was the grim reality our team at RevLock faced when on-boarding our first few customers — none of them could use the data entry UI our team built, and we had to scramble to provide a way for them to actually use our product.

The failure to realize this poses a huge risk for any B2B software product to gain traction.

Traditional Approaches to Solving Data Integration

Typically, software providers attempt to solve this problem using any of the following methods.

  • Building standard integrations for current platforms to cover most customers in the vertical they are targeting.
  • Offering a robust professional services team that generally charges between 1-2x of their subscription to perform the integration for customers.
  • Leaving the problem of loading data for customers, believing that the value their software provides is large enough that customers will figure it out themselves.
  • Creating a custom ETL solution in-house that works for your customers.

However, none of these approaches work particularly well.

  • Building standard integrations is just a dream. We learned this one the hard way — we created standard Salesforce integrations, but we quickly found that most companies of any significant size create processes and data that is custom for them. Even if we got 80% of a customer’s data using a standard mechanism, the remaining 20% required us to use another solution.
  • Offering a professional services team becomes a colossal sales barrier. Most customers want to experience a scale up model, allowing them to try your product prior to finalizing a major purchase decision. Requiring customers to pay a huge sum upfront to integrate disallows this purchase flow — a customer is essentially forced to go all in on your product just to start using it — that’s a hard sell for any sales team.
  • Leaving the problem to customers makes you vulnerable to competition. Every year competition in enterprise software grows tremendously. If you have a unique value prop that makes customers willing to work out the problem themselves, great, but this is rare and isn’t likely to remain true forever. For us, this wasn’t an option.
  • Creating a custom ETL solution takes valuable resources away from developing the core value prop of your solution. After ruling out the above options, this is what our team settled on doing. Although this seems like the best solution, it forces your team to spend time on building something that potential customers may assign little or no value to.

The Embedded Approach

Since building our own ETL solution, we have been on the lookout for something better. After all, we never wanted to create ETL software in addition to our own product in the first place.

It seems like that new solution has finally emerged in the form of embedded ETL tools, which aim to solve the problems created by the aforementioned solutions.

The fundamental difference between an enterprise ETL tool and an embedded ETL tool is that the tool is “embedded” into the product, rather than living outside of it. The product directly interfaces with the ETL tool to retrieve customer data and run transformations.

In many respects, an embedded ETL tool offers the same functionality as creating a custom ETL solution, but without having to actually build it yourself from the ground up.

Embedded ETL tools are also far cheaper than enterprise ETL tools because they run on your infrastructure, so you only pay for what you use. They also tend to be more intuitive and easy to use from a developer standpoint because they leverage widely used open source ecosystems such as that of Python.

Since embedded ETL has come into interest, prominent ETL tools have begun introducing embedded extensions.

Stitch — “a cloud-first, developer-focused platform for rapidly moving data” — introduced Stitch Connect in 2018 to “allow developers to incorporate data integration into their applications” and has been adopted by many software firms including Chartio.

Our team at RevLock decided to trash our custom ETL solution completely and switch to using an embedded ETL tool (if you’re interested, you can read about the details of that process here).

So far, we’ve been pleased with the results:

  • Our team has had more time to focus on developing our product.
  • On-boarding customers has been faster than ever before (and we’re gaining traction because of that).

I would recommend any B2B SaaS firm to consider an embedded approach to ETL as more and more tools are being developed following this model for solving ETL — it might help you as much as it helped us.

Conclusion

Getting customer data into your product is a crucial part of a B2B product.

Solving this issue effectively minimizes the barriers to adoption, enabling you to offer the modern scale-up models that customers are increasingly demanding and gain traction easily.

Traditional ETL solutions fall short for startups, but building custom solutions isn’t a great answer either. Recreating the wheel takes time away from developing your product — a SaaS firm shouldn’t have to build ETL software on top of their own software.

Embedded ETL tools give the effect of a custom solution minus all the hard work, and is a solution worth considering for SaaS B2B firms.

Agile Insider

Exclusive & practical insights about rapidly delivering value.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store