Isolating JUnit Tests for Fun & Profit

Max Mautner
Oct 23, 2018 · 3 min read
Image for post
Image for post
Continuous Integration’s the name of the game!

Where we’re coming from

Testing our software is fundamental to Personal Capital’s culture of product development.

We lean heavily on Jenkins for continuously integrating our engineering teams’ changes to our front and backend software.

On the backend, we use Java’s JUnit + maven-surefire plugin to run a suite of automated tests which verify that our APIs behave as we expect them to.

Things generally go smoothly however there are some tests we have that attempt to test large swaths of our software at once.

These tests often break when small changes are made in our codebase, or worse: they only pass some of the time, making it very difficult to identify sources of uncertainty in our software.

Where we’re headed

In order to expand our CI/CD process, we’re requiring pull requests to pass tests before they can be integrated into our shared codebase.

To accomplish this we need to:

  • eliminate unreliable tests like mentioned above, and
  • cut down the time it takes to run our tests — slow tests are tests that no one runs!

It’s critical for technical and non-technical team members to trust our tests. This means holding new and refactored code to the same high standard of requiring tests to run and pass before any additions or modifications are allowed to be made to the codebase.

In order to accomplish the two goals above, we can use JUnit’s @Category annotation to tag slow test classes and methods. Once tests have been tagged, we can either include or exclude them from our test runs.

This meant creating a simple interface, for example:

package com.personalcapital;public interface SlowTest {}

This interface can be used in combination with the @Category annotation to identify slow test classes:

import com.personalcapital.SlowTest;@Category({SlowTest})
public class FooTest {

}

Or slow methods:

public class BarTest
{
@Test
@Category({SlowTest})
public void testSomethingThatShouldBeMocked() { … }
}

Invoking the test Maven goal, we can run only the slow tests, e.g.:

$ mvn test -Dgroups=”com.personalcapital.SlowTest”

Or exclude all slow tests:

$ mvn test -DexcludeGroups=”com.personalcapital.SlowTest”

We can exclude slow tests when automatically running tests for a proposed change, and run our slow tests later on when we evaluating a manual release to production.

Using Jenkins’ JUnit Plugin, it becomes very easy to identify slow tests by treating your test output as structured data:

What to look forward to

We’re delivering tools for our customers’ to better manage their personal financial lives at a faster rate than ever before, with a larger team than ever before.

We’re striving for the holy grail: continuous delivery, where a team member might join Personal Capital and confidently contribute code to production within hours of walking in the front door.

Come and help us reach that goal!

Personal Capital Tech Blog

We are Personal Capital's Engineering team.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store