I Want to Build in an “Open Way” and “Avoid Lock-in ?”

What does this statement actually mean anymore ?

In the past years, I have helped clients with “Cloud Strategies”, “Microservices Strategies”, “App Modernization Strategies”, or “Dev Ops Strategies” to name a few. There is usually a common theme around being “Open” or “Avoiding Vendor Lock-In.” As I see these clients evolve into implementing something, I see behaviors or choices that I deemed as not really “Open”, although in the eyes of the implementor it is “Open.” So I went through some self examination of what does it mean to be “Open.” As I talk to clients, partners, and colleagues, there are many different definitions. Now I am “not an academic type,” so I am going to take liberties with terms for the intension of understanding. I am going to list a few “answers” to the open question I have received.

Different interpretations of “Open”:

I Build my Application according to some specification
This is a classic answer. In the past, standards like “J2EE” (now called Jakarta EE, after being Java EE for a while). The promise here is if I build my application according to a standard, no matter whatever application server I deployed to, my app would function. So like 15 years ago, if I built my app using Java EE, I was promised that if I switched from WebSphere to WebLogic, I would be fine. The reality of that promise never fully lived up to the hype. It was very narrow in that as it did not look at thinks like: Server configuration, tons of configuration scripts needed to configure the server, all the testing around keeping it up, etc…. Finally, the specifications which was created and maintained by committee moved as slow as a committee, as such, every vendor extended the specification to solve real problems. Ultimately, this notion usually called “application portability” only solved a very narrow definition of the issues. It is worth noting that projects like Microprofile have modernized the Java EE Based programming model.

I Build my Application according to a “Defacto Standard” or “Popular Open Source Project with a Vibrant Community:” 
In this case, someone chooses an open source technology and builds around it. 15 Years ago, two popular Java Open Source frameworks (Spring and Hibernate) changed the Java landscape in that it was deemed more open to build an application using these frameworks because I could develop with a a stubbed out environment and run it anywhere. Spring has had a very long shelf life and has evolved as one of the most popular ways to do Java. However, this approach to “open” can be as fickle as fashion trends. One has to look at JavaScript Frameworks. 15 Years ago, as Ajax and Web 2.0 emerged, so did JavaScript Frameworks, and today we live in an age where they last about 2 years. In the past 10 years, the community went through a bunch of JavaScript Ecosystems: Dojo, jQuery, Bootstrap, Angular, React, and Vue (And many, many more). The other aspect here is finding skills for projects, with the assumption that a vibrant community means i can find developers (or SRE’s) with skills a lot easier.

I want to build only with Open Source:
In this case, a team is building an application, and they want to solely use Open Source Products. The idea is one does not have to pay for product licenses, they can fork the source, and built their own tools. Often times, the development team builds a home grown stack. For example, I may decide that I am building a Kubernetes Environment. That reality is the development team will be integrating dozens (hundreds) of open source products together and manage their interdependencies themselves. For example:

  • Kubernetes Stack from Open Source.
  • A Docker Engine or Containerd
  • A Kubernetes package manager like Helm
  • Automation and DevOps for the Platform (An abstraction around platfrom scripting: Ansible, Cloud Form, Terraform, puppet, chef, etc….)
  • An Application DevOps Stack: 
    - Jenkins (Builds / Deploys),
    - Git (Source Control), 
    - Artifactory (Code, library repository)
    - Tarus (Testing Harness), 
    - JMeter (Test scripts and performance),
    - Gradle/maven/npm (Dependency Management, possiblly multiple per . language)
    - Security Code Scanner, 
     etc…)
  • A Logging Stack (ELK)
  • A Monitoring Stack (Prometheus, Grafana, etc….)
  • A Metering solution
  • Configuration (etcd)
  • Storage (GlusterFS, Ceph, etc…)
  • A bunch of Glue code between all the above
  • Documentation and examples for how development teams can use the stack.
  • How to bridge existing old stuff, deal with VM’s, etc….
  • An API Gateway and Microservices Mesh (Netflix OSS, Istio, Envoy, nginx)
  • Security Packages

Maybe you will get lucky and build the new super platform that evolves into a market leader. I have seen large organizations roll their own platform only to have development teams not adopt it. I did a “Red Team Review” a few years ago where an insurance company built a provisioning framework out of 200+ open source packages and they could no longer evolve it.

I want to build on products based on Open Source: 
This is slightly different, in that companies are looking to reduce risk, but still want the support of a company, and perhaps reduce the complexity of building your own stack and integrating. For example, a product like IBM Cloud Private (ICP) or OpenShift are built on various Open Source Frameworks. Both ICP and OpenShift are built on Kubernetes and several other Open Source Projects. If you stick to certain patterns, you can remain agnostic, however, often times, not 100%, as there are always areas like security, developer productivity where there are gains in speed to market or security features that might work against open, and that may be ok. For example, OpenShift has some opinions that might not agree with the Kubernetes core distribution . For example, OpenShift users may prefer routes over Kubernetes Ingress for inbound traffic.

I want to build my product in Open Source and sell support/services
This is a different point of view, where someone might want to build a product in the open, sometimes combining other open source products. For example, Redhat’s OpenShift is based on OKD, which is an open source distribution in itself. SO even though OpenShift might have an opinionated Kubernetes stack, they contribute those opinions back to the Open Source community reducing some of the risk. It is worth noting that sometimes productivity and getting things deployed through this value add is more important than the open factor.

I want my application to be able to run in any cloud and on-premise if possible.

Today, this seems to be the modern issue with Open, can my applications run on-prem or in any Cloud Provider. At first, this may seem like JUST an application portability issue. The application code might be the easiest part to make portable (although things like Serverless/Lambda might bring this issue back). However, one has to expand a bit to realize the lock-in now occurs AROUND the application:

  • Are the services or features the application uses portable (A Cloud specific database like Dynamo DB on AWS vs. an Open One like postgres, etc…). In addition. In addition, Higher level services (AI services, Watson, etc…) are included in this.
  • Are my deployment and provisioning scripts cloud specific ? If so, rewriting those might be more difficult than the app itself.
  • Will my performance, sizing, and all the testing that goes with it be the same.
  • Will my development process and tools be the same. For example, most clouds have Cloud Specific CI/CD tools. (Azure CI/CD, AWS Pipelines, IBM Cloud Pipelines)

There may be valid reasons to use these services.

Examples:

So how do I protect myself (or minimize) lock-in ?

I am sure there are more definitions out there of what it means to be Open, but given this, what should I really be concerned about if you want to avoid lock-in ?

My application can run in any cloud or on-premise.

Choosing programming languages and stacks I can package and deploy to a cloud. Kubernetes right now has by far the most traction for container platforms, so it is a good choice right now.

Look at the services you use in your application. For example, if you need a JSON Based NOSQL database, something like MongoDB might run in multiple clouds. Look out for issues around API Compatible vs. full featured, looking at the recent example of AWS releasing a “Mongo Compatible API DB” called DocumentDB.

Some services are just SaaS features, the best you can hope for us the ability to “Invoke” those services from anywhere (although with latency added)

My CI/CD Pipeline can deploy and package applications in any cloud.

The center of control for development in a modern enterprise is the DevOps environment (Git, CI/CD, etc…). Using standard Best practices for building Docker Images, deploying with Kubernetes, etc…. is very important.

I can provision or configure (aaS) in a neutral or pluggable way.

How I provision and Configure an environment is another source of “lock-in.” If you are deploying to multiple clouds and on-prem, technologies like Terraform can be an option. But there are many others like Puppet,Chef, and Ansible. etc… Sometimes you might just have to live with having “Multiple” scripting techniques for clouds.

I can extend the platform to any central monitoring, logging, and metering.

This is an often over looked area, in that if you are deploying to multiple clouds, you have to choose between out of the box monitoring vs. having an infrastructure (or using one like Splunk, with some cost). Often times, it is best to compromise here and make sure you have the ability to do both. For example, you can use IBM Cloud Private’s prometheus and Grafana but ALSO forward prometheus data to an external source.

Base it on “Open” technologies that you can easily find skills in.

Using produces based on well known technologies will always help with skills. If you have a Kubernetes based platform for example, a person with skills in one stack can usually move to another with a short time of enablement.

It will usually not be possible avoid “Lock In” 100% . If you are using any Cloud, you might be already. However, think more broadly about your goals and how the term “Open” has a few interpretations.

> Learn more about how you can co-create with the IBM Cloud Garage.