Using Zapier, Node.js, and Firebase Together

ZAP!

Supplementing code with advanced workflow tools

TL;DR

1. Zapier was great for prototyping out the flow for our lead-nurturing solution quickly as an early-stage startup.
2. We had some issues with this solution’s convoluted setup and suboptimal error handling and reporting.
3. Ultimately, instead of just throwing it out completely we took the most useful and reusable parts of Zapier and kept them while developing our more custom solution in a Firebase Cloud Function.

Introduction

Zapier markets itself as an alternative to interacting directly with APIs and a way for non-developers to connect apps without writing code. As an early-stage startup, Zapier was an incredible tool for prototyping our marketing tools. We used it to quickly build a custom lead-nurturing solution for our agents that centered around autoresponders, so they could immediately start growing their businesses under our guidance.

When we decided to build out a more stable version of our prototype with Firebase, we initially assumed that Zapier would be taken out of the equation completely. However, we realized that Zapier has many benefits beyond just connecting apps without having to write any code. We ultimately decided on a hybrid app that would combine Zapier features like …

• Listening for leads
• Formatting, parsing, filtering, and normalizing data
• POSTing data to a Firebase Function custom webhook
• Assisting with testing

… with a Firebase HTTPS Cloud function that would perform all the application logic and data sending and retrieval that Zapier used to do!

The core functionality and the business logic of the solution was moved to Firebase. However, Zapier is still so helpful to us that we thought that we would share our story so that other developers could see how adopting it can benefit their projects when used in this non-standard way.

Supplementing code with advanced workflow tools isn’t a cop-out, it’s smart!

Our Original Zapier Setup

Originally, we used Zapier as it is intended to be used: in a matter of hours we stitched together several services using Zapier’s UI to make the first lead nurturing/management Zap.

The tool we built was an immediate success and several of our agents quickly adopted it. Their feedback resulted in many improvements to the system. Zapier’s flexibility made it such that parts could be quickly added or taken out without worrying that the rest of the system might be affected. There was no pressure to commit to permanent changes because of a significant upfront time investment.

In this phase of our business, the intended benefits of Zapier were evident: it’s fast, flexible, and simple.

Issues We Were Facing

Subsequent changes to the Zaps eventually lessened as we perfected the autoresponder system. Around that point we began to need stability, accountability, and reusability over speed, flexibility, and simplicity. There were several issues in particular that prompted us to look into moving away from Zapier and building our own system from scratch:

• We needed more detailed error reporting at every step of the process. In some cases, Zapier would report that a step in the Zap succeeded because it made contact with an app, not realizing that the response returned an error message.
• If a Zap did fail at any step (either due to bad data or one of the connected apps failing) the Zap would shut down at that step and not attempt to complete the rest of the steps. In an ideal situation, the Zap would try to complete as many of the steps as possible before giving up.
• Setting up Zaps with so many steps and integrations for each agent was time-consuming and prone to human error.

The Solution

What We Kept (and Use Zapier For Now):

Our autoresponder Zaps that were previously 13+ steps have now been cut down to 3 to 5 steps. What the Zap does now is essentially a three-step process:

1. Listen for and receive leads
2. Filter, parse, or format lead data that comes through
3. POST the data to our Firebase function webhook URL!

Even though we did not use Zapier in its intended way (connecting apps), we still gained many benefits from using Zapier in this way. These benefits include:

• Event listening and triggering
• Data normalization and filtering
• Custom behaviors
• Exhaustive list of integration
• Reporting and analytics — easily provide data to those who need it
• Easy data shaping
• Accessibility to non-technical team members
• Assistance with testing. We didn’t have to submit real forms to test and the Zapier Chrome extension made sending a test as easy as clicking a button in the browser.

What We Took Out (& Use Firebase For Now):

The Core Engineering team at Side already uses Firebase:

1. As a database using RTDB and Firestore.
2. To host and run backend code that is triggered by changes to the database or HTTPS requests using Cloud Functions.

We created two Firebase Cloud Functions to handle the bulk of what the old Zaps were previously doing (connecting with, sending data to, and retrieving data from apps/services). In addition to matching the functionality of the previous Zapier autoresponder system, we added some improvements:

1. Integrate with the Side Core App so that we retrieve agent data from the Side database within the Firebase function. This resulted in only needing one Zap for each channel that leads come through (e.g. Zillow, Realtor.com, website form). In this example we would only need three Zaps in the new solution versus 3 * n (where n = number of agents) in the old implementation.
2. Write all ‘interactions’ to a collection in Firestore. With this data, we can identify past leads in other functions and give an agent context about the lead when they respond to the autoresponder. We also hope to use this data in the future to analyze our marketing efforts so we can continue to optimize the results.
3. Write logs of the result of each step of the Firebase Cloud Function (success or failure and the context/error message of either) to a collection in Firestore and then write these results again to a Google Sheet. A row is written to the sheet every time a new lead comes in and if any step of the Function fails, we can easily see where the failure occurred and why it happened.
4. We structured our JavaScript in the Cloud Function so now if a step fails it does not halt the entire function at that step in the process. The failure for the step is reported and the next step in the process is attempted.

Conclusion

Using Zapier in conjunction with Firebase has given us the freedom to make a custom solution with improved error reporting that is more integrated with our company’s database. We love that it has eliminated some of the more boring, error-prone parts of coding. This setup has proved so successful that future Marketing Engineering projects at Side that require listening for and normalizing data will most likely use this same combination of tools.

The state of software development in 2018 is such that UI tools and code are not two separate or opposite ways that one must choose between to get the job done. When used together thoughtfully and with intention, UI tools and code can help engineers save time and focus on the fun and interesting parts of software development.