Separating The Risk Of Vendor Lock-In From Migration Costs
A few weeks ago, we decided at Trellis, the company I work for, to use an Authentication as a Service solution for our user management, and we set out on short journey to select the right vendor for the job. Authentication is a central part of any user facing system, and our our awesome platform is no exception. (By the way, we’re hiring experienced Software Engineers, If you want to work with me, drop me an email!)
Hence, we wanted to make sure the provider we end up picking will the most likely fit for us. But beyond that, we also wanted to make sure that if we end up realizing that we made a bad choice, we’ll be able to switch later — or in other words, we wanted to avoid Vendor Lock-In.
As Software Engineers in a world of cloud solutions, the above is a common scenario. We need to make a choice whether to use a 3rd party service, and we will often have a few possible vendors to choose from, as well as the option of building things in-house. When making the choice, the frightening notion of “Vendor Lock-In” always looms.
While fear of vendor lock-in is not without reason, lately I’ve sensed that this fear is oftentimes exaggerated beyond its right measure. This exaggerated fear increases the chance that you’ll be making costly choices, either directly by picking a more expensive choice, or indirectly by missing out on offered opportunities. Therefore, I believe it is worthwhile thinking about vendor lock-in a bit more — and understand it a bit better. After all, things we understand, we fear less.
Not long ago, Wisen Tanasa of MartinFowler.com wrote a piece about vendor lock-in. As usual, it’s great read, and is recommended. Tanasa’s main point is this: While it is common to analyze vendor lock-in costs as “Vendor Lock-In Cost = Migration Cost”, it is incorrect. According to Tanasa, this would be a correct analysis only from a technical perspective, but one should also make way for a business perspective, and it is in fact more correct to model it as “Lock-In Cost = Migration Cost — Opportunity Gain”. While Opportunity gain is harder to estimate in advance, Tanasa says, the importance of which must be taken into account when making the final decision.
Tanasa makes an important point in that when you consider your technical decision, business considerations must always be present, and even if some costs or gains are hard to estimate they shouldn’t be ignored. But I think that the main problem with Tanasa’s line of thinking is that he, also, conflates too many different meanings into the term “Vendor Lock-In Costs”.
This might seem nitpicky, but as I said earlier, conflating different types of costs makes it hard for us to reason about each one, limits our ability to mitigate risks and reduce the costs that are reducible, and making informed decisions (this is all the more true of conflating gains and costs, but let’s leave that one aside for now).
Tanasa literally equates Migration Costs to Vendor Lock In Costs, and it just begs a reminder that Vendor Lock In and Vendor Migration are not the same thing — A lock In is exactly the situation where you might want to pay the cost and migrate away from a previously selected vendor, but you can’t — because you are locked in.
There are various reasons why a lock-in might occur, some might not even be technical at all. For example, you might be locked-in because of a legal commitment, or due to ‘political’ reasons (e.g. in a large corporate your vendor might be a different department inside the organization, and using a different vendor will be a marketing catastrophe). But as far as technical reasons go, I see three reasons that vendor lock-in might occur:
- The vendor saves your data in a proprietary format.
- You cannot access the data from an arbitrary service (An API access is limited or non-existent)
- you cannot bulk export your data (either not at all, or not into any open-source data format such as JSON).
If any of the three statements above is true for your use case, then you are in serious danger of a vendor lock-in. In my view, unless you have a very strong reason to believe that the above three won’t pose a problem for you, you’d be better avoiding any vendor who does not give you control and portability over your data.
Note though, that proprietary data format is different from proprietary features. Such features, like unique extensions to SQL available only in a specific engine/dialect, or built-in logging/analytics add-on to a cloud service, are never a cause for vendor lock-in. This is because features, unlike data, can be re-vendored or reimplemented at any time. And while costs of buy vs. build can play a role as they always do, the option to build always stand, and in that sense lock-in does not occur. That is, as long as the data is yours, is accessible to you from outside the vendor service, and is universally readable.
This is an especially important point, because I often see a tendency to avoid using vendor-specific features because of fear of vendor lock-in. While the costs of relying on an offered feature (e.g. migration costs, but also maybe just the plain old pricing) might still outweigh the gains, it will never be due to vendor lock-in. Or to put it in different words, if you found yourself relying on a vendor that offers features that are not offered by any other vendor, and to build the same features in house would take too much work — then the vendor doesn’t lock you in, they are just offering a superior solution.
All that being said, it is still worth noting that from a practical point of view, any expensive enough migration, is not different from being locked in. I believe this fact explains more than any other reason why lock-in risk and migration costs are so often conflated. Similarly, locked-in data can also theoretically be migrated or recreated at a cost — rendering any lock-in scenario as just another case of migration costs, and we’re back to Tanasa’s point.
To this I’ll say three things:
First, unlike features or code, data may really be irreplaceable.
Second, even when replaceable, data migration from a service that is inaccessible or relies on a data format that is unreadable, will likely be an order of magnitude costlier than a migration project where the data moves around freely.
And third, in today’s world of Open Source Dominance, abundance of universal data formats, and relative ease of API building, vendor lock-in should never happen, at least not for technical (=data) reasons. Yet, some vendors will still try to trap you in. If you make yourself used to noticing the difference between possible future migration costs and the risk of lock-in, you will have easier time to Just Say No.
In conclusion, Martin Fowler himself will surely agree that the terms and phrases we use to describe our problems should be as exact as possible. In many ways it is like naming variables, or better yet, naming design patterns — the terms are the cognitive handles and levers with which we operate our reasoning about abstract engineering decisions. Having a clearer grasp of when a risk is associated with vendor lock-in, and when it is associated with migration costs, will definitely help me make better choices, and it already has.
I hope it will do the same for you.
