Four Steps to Build an Integration Strategy

Integrations between a central CRM system and multiple workforce management systems.

I once helped a local government replace the legacy software that its non-emergency call centers were using with Salesforce. I gave an overview of what the solution architecture for a project like that looks like in a previous blog post, but there’s also another aspect that’s worth diving into. This was a large city with more than 150 departments that could potentially respond to non-emergency requests. Many of those departments had their own sets of IT systems, which had to exchange data with the city’s new Salesforce org.

The system we replaced was more than 20 years old and its integration capabilities were limited to file-based data transfers and SOAP-based web services. Many external systems weren’t integrated at all, requiring users to update data in multiple places. But since we were going to be replacing that legacy system with a more modern one that had better integration capabilities, the city’s IT leadership decided to use the project as an opportunity to assess its current integrations and create a strategy to improve them.

You may have encountered similar scenarios in your own organization. It’s not uncommon for teams to build ad-hoc integrations that solve specific problems related to an in-flight project or in response to a request from an influential stakeholder. Once these integrations get to production, there usually isn’t much of an incentive to change them, even if they don’t really align to the organization’s overall integration needs. If this happens often enough, an organization will eventually find itself inundated with a set of non-strategic integrations that can create a variety of issues, including:

• Data inconsistencies between source systems and target systems
• Data not arriving at its intended target systems in a timely manner
• Unanticipated downstream issues when fields get updated in a source system
• Security vulnerabilities when sensitive data is left unencrypted while in transit

Over time, these issues can compound to reduce the overall quality of your organization’s data. This becomes an even bigger issue if you want to add artificial intelligence features to your applications. AI requires large volumes of high-quality data to be effective. So if your systems are riddled with data quality issues because you don’t have a handle on your integrations, you’re likely to encounter problems like inaccurate predictions that lead to bad business decisions, or poor results from large language models responding to prompts that contain incomplete or inaccurate data.

So how do you create an integration strategy that can address your existing issues and prevent them from happening again in the future? When I had to solve this issue, I worked with the city’s IT leadership to tackle this challenge in four steps. You can use a version of this approach for your organization.

Step 1: Build an integration catalog

Our first step was to document all of the integrations that already existed in the city’s IT landscape. There are enterprise architecture tools that can help with this, but many of them are paid products that you’ll need to learn how to use. If you want to keep things simple for this step, you only need a spreadsheet. We created one with the following columns and worked with our business and technical stakeholders to populate it:

• Data Element: The data that’s being sent in the integration (for example, accounts)
• Source System: The system the data originates in
• Source System Owner: The individual or team responsible for maintaining the source system
• Target System: The system the data is being sent to, if applicable (some asynchronous integrations may not have a defined target system, or the target systems might be outside of your organization)
• Target System Owner: The individual or team responsible for maintaining the target system (if applicable)
• New Records Per Month: The approximate number of records relevant to this integration being created per month
• Changed Records Per Month: The approximate number of records relevant to this integration being modified per month
• Update Frequency: How often relevant records are updated (some change at random intervals as users interact with the system while others are updated via periodic batch processes)
• Synchronization Type: Indicates whether the source system sends a copy of the entire record, including unchanged fields to any target systems (full copy), or only sends the fields that were modified (delta)
• Trigger: What makes the integration execute (for example, saving a record or a periodic automated process)
• Integration Method: How data is actually transferred (for example, REST API or file transfer)
• Security Concerns: Personally identifiable information, personal health information, or any other data that may be subject to regulations
• Additional Notes: Any other relevant information, including links to more detailed technical documentation about an integration if it exists

In addition to discussing your integrations with stakeholders, populating this spreadsheet may also require you to read through old documentation or even look for clues within the systems themselves if you can’t find someone who knows how an integration works. Keep in mind that this type of research can take some time and you should plan accordingly.

Also keep in mind that it’s okay if you do not have enough information to fill out every field in this list, or if you want to add more fields to capture additional information that isn’t captured here. Your goal at this stage is simply to create a list of integrations that you can review holistically when you move to the next stage, so do whatever is best for your organization.

Step 2: Review your catalog

After we built our catalog, our next step was to review the integrations we cataloged together to identify any potential issues, including:

• Security vulnerabilities: Integrations containing unencrypted personally identifiable information (or other regulated data)
• Unnecessary redundancies: Multiple integrations sending the same data elements to a target system
• Incorrect source systems: Data elements being sent from systems that were not their systems of record
• Outdated integration methods: File transfers or similar methods being used where API calls or other methods would have been a better fit
• Systems with no owner: Lack of clarity around whose job it is to investigate failures

Step 3: Prioritize builds and fixes

Once we identified as many issues as we could, we prioritized them and created a roadmap to address them. We also identified some new integrations that we wanted to build as part of our current project and created new entries for them in our catalog as well.

It’s important to note that we didn’t plan to fix every existing issue as part of our current project. Chances are that you won’t be able to fix all of your organization’s issues right away either. And as long as they’re not causing major business disruptions, you don’t need to. The one exception would be security vulnerabilities, which should be addressed as quickly as possible.

For everything else, your goal at this stage is to create a roadmap that helps your stakeholders understand when they can expect any existing issues to be fixed. You can determine where each item belongs in your roadmap by considering the level of effort required to fix an issue with other attributes you’ve collected, like the number of records affected and frequency of updates to determine the correct priority. Then you can plan to roll out fixes alongside any current projects or plan new projects specifically to address existing issues.

Step 4: Establish a repeatable process

The last thing to keep in mind is that this is not a one-time activity. As you fix all of the issues you’ve identified, be sure to make the appropriate updates in your catalog. You should also require project teams to keep this information up to date if they add, update, or remove an integration during a project.

For my project, we also updated the city’s design standards to include guidelines for what integrations should look like going forward. Then we communicated the updates to the city’s delivery teams and set up a process to review any new integrations to make sure they were in compliance before they could be migrated to a production environment.

It took some time for us to create our strategy and put the right processes in place, but the city was happy with the results, which included a reduction in duplicate data entry, streamlined processes, improved data quality, and a better overall handle on their integrations. It’s likely your organization will see similar results if you follow the approach outlined here.