Snapshot tests or how to stop layouts from breaking (again)

Animation by Adam Plouff

Scenario: a designer asks you to update padding on one of the screens (let’s call it Screen A). Simple enough task (maybe even too simple). In less than a minute you find the layout in question:

<!-- Screen A -->
<LinearLayout
  android:padding="?screenAPadding">
  <!-- Bunch of views -->
</LinearLayout>

Oh, interesting. Someone decided to extract padding into a theme attribute. Interesting (albeit questionable) choice, but no matter. Let’s find where the attribute is defined and update it.

Another minute and we found it:

<style name="MainTheme">
  <item name="screenAPadding">8dp</item>
</style>

Change value to whatever it is designer wants and voila! Everyone is happy…

Problems

… except for a couple of screens which we just broke. It turns out that there was a Screen B which was for whatever reason relying on the same screenAPadding attribute:

<!-- Screen B -->
<RelativeLayout
  android:padding="?screenAPadding">
  <!-- Bunch of views -->
</RelativeLayout>

Ouch. But things don’t stop there. It turns out that screenAPadding attribute has a different value for a dark version of our theme (god knows why):

<style name="MainTheme.Dark">
  <item name="screenAPadding">10dp</item>
</style>

Ouch (again). If you are lucky (or have a diligent QA team), issues like that would be caught early. If you are not, it will slip into production.

Can I not have this problem, please?

Of course! There are several ways to prevent this from happening, in descending order of what it would cost you in terms of time and money:

• Do a sanity check before each release. For each possible layout setup.
• Have a QA team to do that for you.
• Write screenshot tests to do that for QA team.
• Write layout snapshot tests because you don’t like running tests on emulators.

While the first two options are quite straightforward, that is likely not why you opened this article (otherwise I am sorry). Let’s focus on how can we automate the testing process.

Screenshot tests

The idea is simple:

• Take a screenshot of your view (while it is still in a good state).
• Save it. Let’s call this screenshot a “standard”.
• On each test run, take a screenshot of the view and compare it to the pre-saved standard.

Sounds easy enough. In fact, that is exactly what Screenshot Tests For Android library from Facebook is doing.

There are some downsides to this approach:

• Tests need to be executed as instrumentation tests on an actual device.
• Setup is not exactly straightforward. You would need to make sure that every developer in your team has the right environment to run the tests.

None of those is a show stopper if you have enough time and dedication to make it run.

Layout snapshot tests

If you can’t afford to spend time setting up an environment for running screenshot tests (especially when it comes to running tests on a real device) or just not sure if your team is willing to commit spending time on something which might not get traction, there is another solution.

What if instead of capturing screenshots (as in — actual images) we would just capture attributes of the views, such as position, text, colours, etc.? That would solve both problems:

• No need to run tests on a real device as Robolectric can handle that as well.
• No need to set up the environment.

LayoutVerifier library is aimed to solve those two problems.

LayoutVerifier

The idea of LayoutVerifier is very similar to that of a Screenshot Tests:

• Get a snapshot of view attributes (positions, content, etc.)
• Save it to a JSON file.
• On each test run, compare attributes of a view to a pre-saved JSON file.

Unlike with Screenshot Tests, it is enough to add a single dependency (assuming you are already using Robolectric, which you probably are):

testImplementation 'com.redapparat:layoutverifier:+'

And then write a regular unit test and let LayoutVerifier do its magic:

@RunWith(AndroidJUnit4::class)
class ScreenATest {
  lateinit var layoutVerifier: LayoutVerifier

  @Before
  fun setUp() {
      layoutVerifier = LayoutVerifier
          .Builder(getApplicationContext())
          .build()
  }
  @Test
  fun `Default layout`() {
      layoutVerifier.layout(R.layout.screenA)
          .match("screen_a")
  }
  @Test
  fun `Small screen`() {
      layoutVerifier
         .layout(R.layout.screenA)
         .screenSize(400, 600)
         .match("screen_a_small_screen")
  }
}

And it is done. By the end of the test you will end up with a file in your project which would look something like that (oversimplified version):

{
  "schemaVersion": 3,
  "features": {
    "visibility": "VISIBLE",
    "children": [
      {
        "left": 0.0,
        "top": 0.0,
        "right": 10.0,
        "bottom": 41.0,
        "id": "com.redapparat.layoutverifier.tests.test:id/textView_a",
        "visibility": "VISIBLE",
        "text": "TextView A",
        "textColor": "#8a000000",
        "textSize": 14.0
      }
    ]
  }
}

That file is then going to be used as a “standard” (“good”) snapshot of your layout. Make sure to commit it to your repository.

Conclusion

In a nutshell:

• If you have time, resources and don’t mind running tests on a real device — consider using Screenshot Tests as part of your testing pipeline.
• If you don’t have that or just not sure whether it is even worth the investment — consider using LayoutVerifier. You can always switch to Screenshot Tests later.
• If you have a good QA team — good for you :)