Reducing API Calls While Minimizing Risk
Here at Uberflip, we pass data back and forth between 3rd party services all the time with our Marketing Automation Program (M.A.P.) integrations (HubSpot, Eloqua, and Marketo just to name a few). One of the most common bottlenecks in these integrations is the amount of API calls required to update information.
A good example from our platform is updating a contact’s properties based on dynamically named fields.
You have to:
- Ask for the contact property information from the M.A.P.
- Create the property in the M.A.P. if is doesn’t exist, or remove the contact property from the update information so as not to break the update
- Update the contact in the M.A.P.
- Try to update every contact property separately in the M.A.P. so it won’t ruin it for the rest
Just to perform a basic contact update 3 possible, 2 definite, API calls are required.
2–3 API calls doesn’t seem like a lot, but when you’re performing this action, for example, every minute a user is on your site, you’ll want to minimize these calls to reduce server time usage. API calls are heavy because you’re waiting on another resource to finish before you can continue and are susceptible to things like time-outs, and SSL connection resets.
Two rules I’ve set for myself to reduce the amount of API calls are:
- Cache what is re-usable
- Don’t trust the cache
This may seem contradictory so let me explain…
Cache what is re-usable
By caching information from a M.A.P. we can use it to reduce the “ask for the contact property information from the M.A.P.” step. There’s only so many contact properties on a contact and also a limited number that you’ll want to update.
The first time you go to update a contact property, check your cache to see if the property name is in there, and if so use the state saved — if it doesn't exist unset it, if it exists then continue.
But if the contact property isn't cached, make the call the server to see if it exists and cache the response before continuing. This way the next time that property is being used, we can check the cache for the state.
Don’t trust the cache
I follow this rule because of one of the golden programming rules: Don’t trust the user. If you think the user might break it, there will be a user that breaks it.
The information that you cache might change in the near future so we can’t trust it. A user might create the field after it was cached as non-existing. Or a user might delete the field after it cached as existing. In both of these cases, the cache’s state for the contact property will be wrong. This can cause major issues depending on the M.A.P.. Some M.A.P.s will stop an update completely when they don’t recognize a contact property.
There’s three things you’ll want to implement to cover this case:
- A refresh button to clear the cache your user can press when they make changes on the M.A.P. side
- An automatic time-out on the cached fields so that they’ll re-check their state every so often. In doing so, you’re pre-emptively avoiding errors
- When an update fails, check to see if the error failed due to an unknown field. If so, refresh the cache then re-perform the update
The first suggestion is for an ideal world with the ideal user. When a user makes a change they notify you on the change. This won’t always happen.
When you cache all the known data and refresh it every so often, you’re reducing the amount of API calls significantly for frequent actions and you’re also sanitizing the data to reduce errors.
So by allowing the user to notify you of change and also not trusting the user to notify you of changes, you've covered all your bases.