CI/CD and the Wind Tunnel
The wind tunnel was the Wright brothers’ competitive advantage, the CI/CD pipeline should be yours.
I visited the Smithsonian National Air and Space Museum four years ago. The one thing that has stuck with me ever since is the meticulous experiments Wright brothers did to get to their aircraft design, and the methodology they adopted. I am convinced that their competitive advantage was their sublime design process, more than anything else.
What their design process gave them is what your CI/CD pipeline should give you. An easy way to experiment and deploy your designs. Test them, tweak them, perfect them — quickly and with confidence. What the wind tunnel was to the Wright brothers is what CI/CD should be to software engineers.
Wright Brother’s CI/CD Pipeline
The Wright brothers had their doubts about Lilienthal’s data and Smeaton’s coefficient that they were using with the widely accepted lift and drag equations. After designing two gliders that wouldn’t produce the lift those calculations predicted, they thought it was time they question the data they depended upon.
The Wrights didn’t invent the wind tunnel, but they were the first to use a wind tunnel as an integral part of their aircraft design process.
They focused on making a miniature wind tunnel where they could efficiently and accurately collect data for potential wing shapes. This allowed them to pick the most efficient wing for the aircraft they wanted to build. Not only were they able to check the accuracy of Lilienthal’s table of coefficients for his wing shape, but they were able to get data for dozens of other shapes. Finally, they corrected the Smeaton’s coefficient of 0.005 that was being used since the 18th century and came up with a new average value of 0.0033. Modern aerodynamics has proven this value to be accurate within a few percent. The key thing to note here is that the Wrights weren’t the inventors of the wind tunnel, but they were the first to use a wind tunnel as an integral part of their aircraft design process.
We may not think of “agile” in the physical world, but that’s exactly what this setup was.
CI/CD is not simply a way to package up artifacts but should offer an environment that allows prod parity with complete verifiability.
Wright brothers never went to college and Orville never graduated high school. However, what they had in their wind tunnel was nothing short of what we should aspire in a sophisticated CI/CD process. CI/CD is not simply a way to package up artifacts but should offer an environment that allows prod parity with complete verifiability.
Scientific method and CI/CD
As engineers, we need to keep in mind the scientific method, you know the experimental technique employed by the sciences that we all probably studied as early as elementary/middle school. Let’s quickly revisit this method.
- Ask a question
- Formulate hypothesis
- Design experiment to test the hypothesis
- Conduct the experiment
- Compare your observations (what actually happened) with what should have happened if the hypothesis holds
- Draw conclusions and accept/reject the hypothesis
- Repeat and profit
Now, ask yourself if your CI/CD environment equips you with the needed tools to follow the above intuitive steps of the scientific method.
CI/CD should serve us as the mechanism to lower the cost of experimentation. Just like the wind tunnel, the CI/CD environment offered to each developer pays off dividends in primarily two ways (if not more):
- The reduced cost of a null hypothesis or an invalid hypothesis that, otherwise, may present itself in the form of service degradation or worse, complete downtime.
- The other bigger intangible benefit will come in the form of a low barrier to innovative ideas. There is no way Wright brothers would have collected data on dozens of wing shapes if the only way to do it was to manufacture gliders. It would have been foolish for them to experiment with non-traditional ideas as the cost of failure would be too high. So, make a wind tunnel for your team, and unleash the innovation by following the demonstrably successful scientific method.
A Word about Microservices and Containerization
A personal dev environment for each developer that closely resembles the production environment is imperative to be able to cultivate a culture of following the scientific methodology. Containerization and orchestration tools such as Kubernetes bring us much closer to this reality than ever before. The complexity to spin up new environments have drastically gone down over the years so much so that a shared “dev” environment can be replaced by individual environments customized to each developer branch. The various functional, integration, and performance tests can be directed towards any custom environment to compare the results.
Depending on the graph of your microservices, your product can quickly become a complicated platform as a whole with several edges that can break things in unexpected ways. Microservices development can be both complicated and simple simultaneously. On the face of it, every microservice is independently deployable and scalable, which may decrease the development complexity. However, as the graph of microservices expands, it becomes harder to reproduce the dev environments locally. The end to end picture requires us to synchronize the deployment of multiple services. Some of these microservices would be the domain of a single team, while others may be managed by others. The dependent services that are managed by others could be consumed from a shared “integration” environment while the services under test would be deployed in their custom namespaces.
Jenkins-X (https://jenkins-x.io/) is a good evolutionary step in this direction. Check it out, if you haven’t already done so. I do think there is room for improvement there, but for the most part, it should solve the following:
- Remote debugging tools: Easy way to open up a port that a remote debugger is able to attach to.
- Rsync files or binaries to the dev environment from the local file system. Make it super easy and fast to deploy changes to your environment while sticking to your favorite IDEs locally.
- Tools to kick-off long-running tests: Scripts that make it easy to kick off tests from CLI.
Conclusion
The one thing that seems to be common among all smart engineers who seem to always solve a sticky issue is rarely extra information that they possess and others don’t. Almost always, it is the process they have with which they approach a problem. They have a knack of creating a tight iterative cycle where they can reproduce an issue and then keep tightening the loop until they are closely monitoring the root cause of the issue. This requires a fairly organized flow of thoughts. They put in logs if the data is lacking. They put in metrics and wait until they can validate all assumptions. In other words, they will always create an environment that allows them to experiment quickly and accurately. Demand that from your CI/CD environment. Take a page from history and make CI/CD your competitive advantage.
