Keep your Enterprise Apps Humming

Real-time and stateful data sync between Salesforce and other Enterprise Applications with Grainite

In just over two decades, Salesforce has become the dominant tool of choice for enterprises to store their CRM data. Multiple other systems such as Enterprise Resource Planning (ERP), Business Intelligence (BI), Human Resource Management (HRM), Marketing Automation, Business Process Automation, etc. need to synchronize data with Salesforce. When a customer record is added or updated, syncing in real-time across these systems allows enterprises to build applications based on a single source of truth, and offer a better experience to customers.

The reality however is that syncing data between Salesforce and 3rd party applications in real-time is loaded with issues. The negative outcomes include

• Businesses find it hard to innovate and release new capabilities
• IT teams spend countless hours trying to fix these mundane yet critical issues

In this blog, we cover

• The methods enterprises take today to connect Salesforce with Marketing Automation Platforms such as Marketo, Hubspot, Klaviyo, etc.
• How these methods fail in enabling real-time sync and become painful to manage
• The need for a stateful system like Grainite in the middle that can manage real-time communication and guarantee delivery of every update

Syncing data in real time between Salesforce and 3rd party applications is loaded with issues

Approaches taken by organizations today

Organizations undertake multiple methods to create a sync between Salesforce and other downstream systems

1. Leverage connectors built by application vendors — Marketo, Hubspot, and other vendors offer native connectors that poll data from Salesforce and send it to these systems
2. Use external connectors made by middleware vendors such as Mulesoft, Tibco, Boomi, etc.
3. Write custom code or cron jobs that read Salesforce data at regular intervals, save the data in intermediate storage, and send it to downstream systems

Challenges with these approaches

The data sync achieved via custom connectors is not real-time. In some cases, the delay could be 15 mins or more. While some external connectors offer real-time data sync, not all of them offer a way to poll Salesforce. Salesforce needs to push the data to them via an agent. This might load your Salesforce instance and make it slow. Another disadvantage of both these approaches is that they are stateless. This has some real business implications

• In case of any failures or system outages, the data changes are lost in transit
• Guaranteeing that the message is sent exactly-once becomes very difficult.
• The development team has to write extensive logic to handle message failures and retries which increases the complexity of these integrations
• Doing any sort of transformation on this data will also require a lot of manual coding

A huge disadvantage of current approaches is that they are stateless

Real-time message sync with Grainite

Grainite is an integrated event processing and data platform that makes it significantly easier to build streaming real-time applications. Grainite is a stateful, strongly consistent streaming application platform built from the ground up, that integrates data ingestion, event processing, data materialization, and storage into a single Kubernetes-managed cluster. Building applications that perform a real-time two-way synchronization between arbitrary sources and sinks is easy with Grainite.

To get data ingested into Grainite, one may either push changed data to Grainite or use the ‘Tasks’ capability to pull data from any offset-capable source in an exactly-once way.

Salesforce to any App: The magic behind Grainite

1. Grainite subscribes to Salesforce’s Change Data Single-Entity channels using our Tasks capability for each of Leads, Account, Opportunity, Contact, and other records of interest to the customer
2. These tasks run continuously and check the Salesforce channel for any new events. New events are automatically fetched from Salesforce as they arrive
3. Grainite transforms the Salesforce data into the required format and sends it to the downstream application using that application’s API
4. The sync in the opposite direction works in the same way

To learn more about the technical implementation, please request a demo

Benefits of Grainite

Development simplicity: Developers can just focus on the business logic. The Grainite platform takes care of the resiliency & scaling for your needs

Real-time sync in both directions: Events are immediately processed as soon as they are available on the source systems.

No message loss: All events coming in from Salesforce are processed exactly once, and full audit logs can be generated.

Automatic handling of retry and failures: Grainite’s action handlers and tasks are resilient to temporary or longer-term failures in any of these systems. Grainite has built-in exponential backoffs to handle any rate limits and network outages.

Real-life outcomes

• Any new lead record entered into Salesforce immediately syncs into Marketo and enables the marketing teams to run the targeted campaigns toward these leads
• An entry in a support system like Zendesk can be updated in real-time to Salesforce. This helps the sales team be aware of ongoing support issues with the account
• No more accidental over-writing of records when multiple reps try to update the account information

These outcomes are not just limited to Salesforce. Enabling a stateful real-time sync between any two external applications becomes extremely simple with Grainite.

About Grainite

Founded in 2019 by leaders from Google and Citrix, Grainite is backed by Sequoia Capital and Menlo Ventures. The team has over 200+ years of collective experience in building large-scale enterprise applications at leading data and storage companies. For more information, check our website or reach us at info@grainite.com to schedule a demo.