Re-Inventing Workflow Automation For The World Of Data
Workflow automation companies altogether earn $10B in revenues a year helping other businesses automate their routine tasks.
Recently, we’ve noticed a trend of people building workflow automations directly on top of their data stack. This is notable because it’s a different ‘shape’ of automation from existing models.
In the 2000s, workflow automations focused on predefined business process steps. In the 2010s, automations revolved around events — effectively SaaS tools firing events to each other. In the past couple years, people have been investing more time into automations triggered by data condition thresholds.
We hypothesize it’s due to the rise of modern data warehouses and roles like RevOps & analytics engineering becoming more popular. In addition, there is an increasing number of data collection tools, much of which gets sent into either Salesforce or Snowflake/Redshift/BigQuery, to facilitate the change of software business models to PLG style sales motions.
Companies are racing to be more data-driven, yet most front-line teams don’t need fancy BI dashboards… only how to focus on the tasks at hand. The best way to achieve this is still through workflow automation channels.
This trend generates new pains, though. A ‘paradigm shift’ is too dramatic a word, but many existing approaches (like Zapier, Tray, Workato) may need to be rethought.
1. Scale: looping through thousands (or more) of rows
Instead of passively listening for inbound events, you now have to loop through entire tables evaluating for logical conditions. Imagine you have 5K customers and want to target high-growth accounts with an outbound campaign. Every day, you have to monitor for accounts that have become high-growth.
The nitty-gritty of this gets quite annoying: dealing with pagination, throttling, batching, etc. You’d really need a technical resource to pull it off, and even then the system gets brittle because you can’t control the order of operations.
2. Business complexity: traversing across multiple business entities
Oftentimes you want to jump between parent-child objects or even the same object from different systems. For example, creating tasks on an account or assigning to a rep based on order volumes for product lines in the account. This is quite a common ‘wish list’ item for revenue orgs.
Many workflow automation tools support SQL for part of this, but it’s extremely inconvenient because SQL doesn’t help you know what the parent of an entity is. This is where Salesforce Process Builder shines more, but of course you lose out on a lot of analytical capability.
3. Analytics: creating actionable context around raw data
Most raw data doesn’t have much business context. The data needs to be benchmarked against time, across segments, and between custom ratios to provide key business signals. For example, one commonly requested signal is ‘get me a list of free trial users that has grown slower than its cohort this quarter’.
While many workflow automation tools support formulas, simple formulas alone don’t help when they can’t be pointed to entire tables/arrays or chained together, like in Excel. Theoretically it’s possible to do this in a more enterprise-grade tool like Workato, but the workarounds are trying.
4. Testing & experimentation: seeing the data side-by-side with workflows
This is not a technical constraint but a UX one. Many DRIs of these initiatives are Operations people. In other words, people who are very very used to Excel. You have different sheets, each sheet with tables of data. You can run vlookups and index-joins and countifs. Filters are common.
These conveniences make it easy to preview, test, sanity check, and experiment with workflows very quickly. It’s an important reason why spreadsheets are so dominant in data analytics. Ideally, a workflow automation tool can recreate some of these interfaces.
Clearly, there are big limitations to what traditional workflow automation tools can do — they’re not meant to be data tools. The other way we’ve seen people trying to solve this is through BI (Tableau, Looker, Metabase). The persistent problem here is BI revolves around visualizations and filters. They’re not really meant to drive workflows. Specifically:
1. State management: seeing changes in records or trigger conditions
A key part of ‘actionability’ is looking at change. Reps are attracted to the ‘new’ notification because this enables them to be proactive and provide a better customer experience. For example, it’s useless to get a list of all users in red health — you want to know the ones that were added yesterday or removed in the past month. That’s critical to running the business.
This can only be done by timestamping whether an account has entered certain states. Clearly, since most BI tools operate on filters, it’s not something that can be done. The only workaround is sending the same report regularly and relying on the end user to keep track of changes.
2. Business logic and variables: a catch-all for the lack of automation capabilities
To the extent a workflow relies on logical branching, case statements, or dynamic content generation, clearly this gets hard with a BI tool. For example, sending a custom email alert to a specific rep or team based on an account that gets triggered is out of the question in Tableau, but very useful to reduce noise.
Finally, anything beyond Slack or email — audience syncing, object creation-like tasks, bulk updating of fields — are all out of the scope of traditional BI as well.
The pains articulated in this blog are ones we hear of every day. This is still true for people who have purchased tools like Tray, are running Tableau, and have tried and failed with Process Builder. Oftentimes people are solving this with 4–5 different stakeholders: a SFDC engineer, a data engineer, a BizTech professional, and an operations manager. This gets very expensive, it makes it hard to experiment and maintain, and projects oftentimes get delayed.
Workbase is working hard on filling these gaps with a familiar notebook and spreadsheet-style interface. It’s an extremely interesting problem — how to bridge the data analyst world with the business technology operations world. We’re super excited about what we’re doing — please let us know if these are pains you are facing in your business and reach out if interested in trying WB!
