Debugging weird errors

Matthew Glover
Aug 9, 2017 · 3 min read

Yesterday I came across a particularly hard bug to solve: I have built a GUI in JavaFX for my Tic Tac Toe application. The first version used standard JavaFX buttons to represent the squares. Which, whilst easy to work with and fine to get up and running, do not look good enough to justify building a GUI.

I therefore decided to replace the buttons with a custom TileUI class. The specific implementation details of the class are not important. But, of course I used TDD to implement it, so knew the features of the tile class worked.

Uncovering the problem

The problem came when I tried to switch out the buttons in the BoardUI class for the new TileUI. This involved replacing a GridPane with a Pane. This simple switch led to all of my BoardUI tests failing. This was hard to understand: only two things had changed: (1) the BoardUI was using a different JavaFX layout, and (2) the Button had been replaced with a custom TileUI class which I knew worked in isolation.

After some investigation I realised my tests were failing because part of the test involved clicking on the first square in the board, but it was now clicking on the fifth square. This was extremely hard to understand, as I was using a TestFX node lookup method which was searching for a node by id, and the id’s of the new TileUI component were the same as the old Buttons, so I could not understand why it was selecting a different button. Also, if there was something wrong with the lookup, I would have expected the click to fail entirely, not select a different square.

Finding the solution

After much trial and error I still had no clue what was causing this problem. In fact, it was only after stopping and going for a coffee that a thought occurred to me, which was how does TestFX implement it’s click handlers?

What I knew, was that when the test runs, a window pops open and the mouse moves to the specified node on screen and clicks it. All of which had worked fine until now. Then I realised that my new TileUI and updated BoardUI were creating a bigger board, whilst my test window remained the same size (and smaller than the new board).

What this meant was that, whilst TestFX could find the node on the UI, because it wasn’t visible on screen, it just clicked the top left corner of the window. This just happened to be in the fifth square (a result of the way the board was being centred). The solution was simple: to increase the size of the window in the test to be bigger than the Board. This was a two second job, which had taken a long, long time to uncover!

Don’t blame your tools!

The easy response to this kind of problem is to blame the limited documentation for TestFX, or the complexity and inherent flakiness of UI tests. And there is some truth in both of these points: TestFX does not have the level of documentation or third party materials that other unit testing libraries have, and there is much more setup involved in testing a UI component than for a simple unit of code.

However, ultimately this doesn’t get me anywhere. Unless I were to conclude that UI tests are not worth doing. But I think this is a cop out. Whilst they are harder, it doesn’t equate that they are less valuable. If testing a function which takes input and output is worthwhile, then why is it not worthwhile to test a component that takes user input? After all, who are we building software for, if not the user? It seems to me that not testing UI code suggests the user is not as important as the code.

The real mistake

In fact the real problem occurred to me as I was writing this blog post. The real culprit was me, which was revealed when I wrote: “only two things had changed”. The problem with this statement is that changing two things, is in fact, changing one thing too many. If I had replaced the GridPane with a Pane in the BoardUI first, I would have uncovered the problem earlier, and would have known the problem had nothing to do with my new TileUI. This would have decreased the possible causes and so made debugging easier.

This was an extremely frustrating and painful lesson. Let’s hope it sticks!

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade