3 Problems Your Non-Automated Builds and Deployments Are Causing You

Mark Bentivegna
Capital One Tech
Published in
5 min readOct 22, 2019

And What You Can Do About It

Metal numbered stamps — 1, 2, and 3

Web 2.0, Adobe Flash, microservices, and serverless architecture. At one point in time, each of these buzzwords have taken the tech world by storm. But how do we separate the pretenders from the contenders when it comes to marked change in the development process?

One important buzzword that many enterprises have been grappling with over the past five years is CI/CD — or continuous integration and continuous delivery — the process of automating builds, packages, and tests for frequent and quick releases. As an engineer who has led the transition to CI/CD in several organizations, I can personally attest to the benefits of the process. I have seen CI/CD help developers, product owners, and leadership solve major problems; some that they didn’t even know they had.

Just getting started in your CI/CD quest? Here is a list of three problems that non-automated builds and deployments were causing us and how we solved them!

1. You Are Wasting Your Time

Having developers manually perform tasks like building, testing, and deploying their code is time that could be spent building features to better serve customers. Before CI/CD practices were prevalent at large enterprises, I had worked on teams where three days out of every two- week sprint were spent on manual regression tests to ensure that new changes didn’t break the existing application. These routine processes were extremely important because they helped us ensure that our applications were of the highest quality., But they also represented a significant use of time. Manually running them meant we were only spending 70% of our time on feature development, effectively reducing our value to customers by 30% because we were doing routine processes manually. By automating our builds, tests, and deployments, these manual release tasks were eliminated entirely and run without any technical overhead. By issuing releases at the press of a button we could now spend more of our time developing code and delivering value to our customers.

The moral of the story is engineering time spent developing is much more cost effective than engineering time spent testing and deploying. Releasing developers from spending time on manual tasks improves cost efficiency by ensuring that engineering dollars are spent on improving your application and your ability to serve your customers.

2. You Are Not Getting Enough Feedback

Crystal Pepsi, Windows Vista, and the Fire Phone were all products that cost millions of dollars to develop. But, despite their differences, they all had one critical flaw in common: customers didn’t want them. By not getting customer feedback on features in development, how can you be sure that you’re building something they actually want? By automating releases you can get your product to market in days instead of months, getting immediate feedback from your customers to see if you are developing something they actually want. Reliable feedback can help prevent very costly and disastrous errors like New Coke and its infamous flawed taste tests.

How could these costly mistakes have been prevented? If these companies solicited enough user feedback throughout the development process they would have realized that consumers simply weren’t interested in their product. While automated releases do not necessarily result in immediate user feedback, it greatly enhances your ability to accomplish this goal. Instead of spending millions of dollars on development and waiting a year for your product to get to market, you can perform iterative releases on a daily or weekly basis to get user feedback without any additional technical overhead. Without any manual builds, manual testing, or deployment approval delays you can rapidly get your application in front of customers to understand if it addresses their needs. Getting customer feedback is more important than ever in a quickly evolving tech landscape and CI/CD practices allow for the quickest release cycles and the fastest way to get feedback.

3. You Are Running the Risk of Human Error

In 1998, NASA’s $125 million craft the Mars Climate Orbiter was lost in space. How could such a costly project fail? One fatal flaw: human error. A developer working on the landing software failed to make a simple conversion from standard to metric units of distance, thereby introducing a bug. Such a large and complex project would have had many test cases to account for, far beyond the ability of humans to reliably execute on every single one, so it’s not entirely surprising that a few errors were overlooked.

As developers, how do we address human error? By having our tests executed automatically! Humans make mistakes but machines, if given the proper instructions, do not. Automated test cases are a great way to ensure that your code is of the highest quality and can help you identify bugs much faster than manual review can. By not having to manually run our test cases, we can perform more thorough and quicker testing, while worrying less about how bugs will impact the quality of our software.

I once worked on a team where we were building an internal-facing application to help monitor business expenses and we uncovered a bug because our tests were manually executed and only covered a few test cases. We encountered an error when a new employee tried signing in and saw an indefinite loading screen with no data being presented. What was the problem? We didn’t test our application to see whether what would happen if a brand new employee, who didn’t exist in our internal database, who signed in who didn’t exist in our internal database. We hadn’t performed tests to include all possible inputs and this very preventable error wouldn’t have happened had we followed proper CI/CD practices and designed automated test cases covering edge cases that were executed with each build instead of manually testing a few possible inputs. Needless to say, we didn’t encounter that error again!

CI/CD and You — A Path Forward

In my experience as an engineer, I’ve seen automating builds and deployments solve many development problems, some we didn’t even know we had! CI/CD allows developers and engineers to eliminate manual tasks and focus on developing features for our customers. It also allows product and leadership teams to get more rapid feedback, improve engineering cost efficiency, reduce the risk of human error in software, and get products to market faster. CI/CD has afforded my teams, and our customers, a much more positive experience; I encourage exploring it for your own organization.

DISCLOSURE STATEMENT: © 2019 Capital One. Opinions are those of the individual author. Unless noted otherwise in this post, Capital One is not affiliated with, nor endorsed by, any of the companies mentioned. All trademarks and other intellectual property used or displayed are property of their respective owners.

--

--

Mark Bentivegna
Capital One Tech

Data scientist with a passion for predictive modeling and big data.