Go Faster, Go Local

Hi all, this will be the first article in our series on delivery excellence. Our delivery excellence focuses on principles and practices which enable agility, engineering and operational excellence within our products.

In this article I will introduce, how you can truly shift left, getting rapid feedback while working with platforms like Kubernetes and frameworks like Spring. I will introduce several frameworks which have helped us move the feedback needle to increase developer productivity on new and existing projects.

Ok let’s dive in but first some context.

Not so long ago, enabling your Continuous integration (CI) process via your build orchestration tool of choice was a sign of success and a maturing journey on the road to enabling fast feedback and building a rapid pathway to production.

Although the introduction of CI practices has been paramount in shifting the delivery maturity needle the world over, it is easy to become over reliant on it to give feedback regarding changes.

So what’s the problem? A key challenge is it can take a relatively long time — minutes, even tens of minutes to understand if your change was successful. Now if you multiply this wait time by the number of commits a day and include the time to merge the commits from multiple developers, the wait time for feedback rapidly stacks up — which is of course is wasted time.

This is was one of the key reasons we began to focus on maturing the developer’s workflow locally — that is on each of their machines. Through this initiative we wanted to achieve the following outcomes.

Increase productivity

Significantly reduce the deployment and testing phases and provide an immediate feedback loop to increase developer productivity.

Test changes in seconds

No waiting for full image builds and deploys. Use frameworks which synchronizes your code directly into your locally running container orchestrator.

Simplified configuration management

Allow engineers to work locally as though the application was running in the remote environment, closing the gap between remote and local development configuration.

Simplified experience

Bring an integrated experience into development environment to shorten the learning curve for developers working with Kubernetes’ and native stacks.

Pain free debugging

Removing the need for engineers to run their local instances against development or staging environments where debugging them can be challenging.

Let’s look at the foundational tools and frameworks that helped us achieve these outcomes.

Local Workflow

To achieve this fast and immediate feedback, we wanted a framework which would handle the workflow of building, pushing and deploying our applications to Kubernetes both local and remote.

Luckily there is a huge open source community dedicated to solving this problem. Two great examples are Draft and Skaffold. These tools automate the building, pushing and deployment of your applications within a pluggable architecture to enable you to drop in the tooling stack of your choice, executed in a hands-free way.

Let’s look at Skaffold as this is what we have been using.

Skaffold collects source code in your project, builds, tags and pushes the resulting artifact to a repository and then finally deploying it to your Kubernetes cluster, all automatically.

If you couple this with a local Kubernetes cluster — like Docker desktop or minikube, you will have the foundation of a fast-local workflow. Simple and very cool!

Let’s take a basic Spring boot application and what a basic stack would look like. The diagram below shows the workflow stages of Skaffold and the frameworks which can be plugged in across the workflow. Highlighted in green is a basic example of just one path your application can take on its way to Kubernetes.

As you can see it’s easy to mix and match your existing stacks into the framework. It is therefore easy to introduce into green field and existing projects with minimum impact to your working software chain.

As we wanted to reproduce and deploy quickly, we introduced Jib to build our images. Jib builds optimized Docker and OCI images for our Java applications without a Docker daemon, you can choose to use Maven, Gradle or library to accomplish this. Jib separates the application into multiple layers so it’s very granular with only your changes rebuilt and not your entire application — making it very fast while ensuring reproducibility.

IDE integration

If you are a user of IntelliJ or VS code, the Code Cloud plugin can make Skaffold integration even more seamless.

The Code Cloud plugin, enables you to set up profiles that are automatically run from within your IDE, no more Skaffold commands. This enables a seamless experience for the developer who can make a change and the plugin will do the rest. The plugin also has some nice features enabling you to see the deployment details — including pod data, right from inside the IDE. Very neat.

Often overlooked, IDE plugins can provide a great way to automate feedback as you code and thus play an important role in the feedback loop — be it linting, security or code quality to name a few. There is a broad range of plugins available regardless of your IDE of choice which enable real time feedback, it’s a simple step which can shorten the feedback loop for every developer.

The CI challenge!

One of the key changes we did have to make when we introduced the Skaffold workflow was rethinking our CI. It was paramount that the process locally mirrored the remote processes otherwise a developer could commit code and get inconsistent behavior and limit the advantages of the local workflow.

To solve this issue we introduced Skaffold into our remote CI process — rewriting our CI process to use Skaffold to drive the automation in our build orchestrator. This does require investment of time but ensures consistency and you benefit fully from your local setup.

It’s all about the tests

Another key ingredient to maturing our local setup was focusing on effective testing locally. This is a broad topic and one we will dive into more deeply in future articles but here I’ll touch on two key areas — Contract testing and effective mocking.

Contract based testing

Ever mocked an API call and the behavior changed, breaking your application? You will know the frustration and how it can add additional complexity and brittle tests into your pipelines. Enter contract-based testing.

There are two common approaches Consumer driven contracts (CDC) and Producer driven contracts (PDC). We started with PDC as it is easier to get started with and adopt. With PDC you test against exposed contracts by the producer which is generally an artifact you pull from a repository. Essentially a stub which emulates the behavior which the producer has exposed to confirm the shared understanding between both parties.

If you are using Spring, you can leverage Spring Contract coupled with frameworks like wiremock which make integration easy, frameworks like PACT expand this concept to a range of languages with a strong focus on the Consumer driven contract principles. Spring also integrates well with PACT broker — so easy to combine the two.

Leveraging contract testing — helped build more reliable and faster integration tests removing brittle dependencies and making the tests easily run within our local environment using Gradle or Maven.

Effective mocking

Mocking has been a stable in the testing toolbox for many years and when working with API’s its often overlooked in favor of live endpoint testing. Hitting live endpoints can make your tests very brittle and require complex data management of downstream services. Therefore, if you want fast reliable feedback effective mocking of your external dependencies is paramount. Frameworks like wiremock easily integrate with your existing code base and integrate easily with complimentary frameworks like Spring contract. It does have its challenges but shouldn’t be overlooked in the pursuit of fast feedback.

Challenges

Getting a foundational local set up working on your development machine is straight forward but how do you drive these concepts across your team and beyond, what type of challenges may you face?

It’s a change in behavior

One of the biggest challenges to scaling these practices across your team and organization is it’s a shift in behavior. A lot of engineers rely on the CI process for feedback and consider this to be working well — others may only be beginning to mature in these processes. In addition, the required time investment can appear unwarranted and it requires continual investment to evolve and mature.

Work with your stakeholders — developers, managers and leaders in your organization to clearly understand the benefits — use examples and working software to demonstrate. Its going to take time, start small, increment and scale as your process and understanding matures.

Education

Outside the frameworks and tooling to establish your local workflow, a key enabler to maturity in this space is being laser focused on establishing effective testing practices coupled with the associated implementations. We touched on key areas like mocking and contract testing but having a solid quality corridor within your process goes beyond these two concepts and its therefore important to ensure your local development practices transverse across the testing pyramid.

What's next?

We will extend the local environment to integrate your Kubernetes workflow with a flexible local version of popular cloud native services — stay tuned for the next installment.

#fidelityassociate