Ghost Inspector

Ghost Inspector for Web UI Testing

Sometimes you need a knife, not a Swiss-army knife. Too many bells and whistles can actually get in the way. For example, our web portal testing requires a recorder to handle straight-forward scenarios: clicking, entering text, and checking state. We also need a way to augment recordings for more complex scenarios.

Most web testing tools can get you this far. But we also need tests to run in the cloud, right out of the box. We want test results sent via email or Slack without jumping through hoops (and certainly without writing custom interfaces). And finally, we want test scheduling built-in.

Ghost Inspector ticks off these requirements and a few more. Ghost Inspector’s Chrome plugin records on-screen operations and assertions. Here’s a quick walk-through of recording an operation and making an assertion. After clicking the Start Recording button, you can simply work in the browser directly. I haven’t noticed any performance lag or overhead from Ghost Inspector while recording.

Ghost Inspector remembers each step until you click I’m Finished Recording. You can switch between recording operations and making assertions from the Chrome extension’s menu at any time.

When you finish the recording, the new test runs automatically, giving you a baseline. By default, tests execute in a headless browser (PhantomJS) running in the cloud.

In this example, I clicked on a “Features” link, and then verified that a heading element contained text “Product Features”. As you can see from the screenshot, the test steps have a CSS selector and an operation.

After the initial recording, you can edit or add new steps manually. The top line is a CSS selector or you can use XPath to identify elements. The wait timeout for each element is set at the test level and can pushed up to a max of 60 seconds. You can also add a Pause operation for any number of milliseconds. Unlike the built-in waiting for an element, a pause will always take the same amount of time, even if an element shows up sooner.

Note: Tests fail if they run over 10 minutes. Both the 60 second and 10 minute hard limits are fine with us — a request that takes over 60 seconds is already in trouble. Likewise with 10 minute tests. A 10 minute test is either trying to test to much, or the application is under-performing.

You can run an operation against elements by selecting from the drop down list.

The list of operations is good for most navigate-and-validate scenarios, but seems a little small for the range of actions you might need. What if I need to do something not covered in the list? You can execute arbitrary JavaScript to break out on your own. Execute JavaScript simply runs without taking a result into account, JavaScript returns true acts as an assertion (a false return value fails the test step), and Extract from JavaScript stores a return value in a variable.

Also notice the “Import steps from test” operation. You can use this to “lego-block” small tests together. Tests that import other tests can be nested up to five deep.

Video and Screenshots

Ghost Inspector records a video during the test run. The video player is light-weight, so it’s easy to move the playback to different parts of the test (my favorite feature). The video playback is condensed. The little example above runs in 13 seconds, but the video is only 3 seconds. If a test fails, the video simply stops at or before the failing step. Other than seeing the state of the page, there’s not much debugging information to be got from the video.

Note: The failing step will list the reason for failure.

A single screenshot is captured at the end of a test. By default, screenshots are of the entire screen, but you can optionally define a CSS selector to target a specific element instead.

Integration and Notifications

We’re auditioning Ghost Inspector with slack and have been happy with the results so far. A moment after a test run, Ghost Inspector posts a message with the test results, the screenshot, and if the screenshot matches the baseline screenshot. This is a great feature that keeps the development team abreast of both test design and results as they occur.

Currently, Ghost Inspector integrates with HipChat, slack, GitHub, Jenkins and several others. You can also use the Ghost Inspector API to integrate suites and tests into your workflow. Calls are relatively simple GET requests that pass the API key included with your account. Here’s a sample of the API:

Scheduling

Even the free account has a set of simple schedule options for daily, hourly, 30–15–5 minute intervals while paid plans allow detailed schedules. When you select an option, Ghost Inspector estimates of the number of test runs you will consume. This can be important because Ghost Inspector pricing plans by the test run, not by the amount of time or other resources consumed. Here’s a screenshot of their pricing plans as of this writing:

So Whaddaya Think?

Web testing platforms have significant trade-offs in complexity, usability, test run performance, test debugging, and integration with other systems. So far, Ghost Inspector appears to be “right-sized” for many web UI testing tasks. It uses a consistent element identification standard (CSS and optionally, XPath), records and executes quickly, and integration configuration has been straightforward. Anecdotally, the support questions I had were answered in a day, and in one case, within an hour.

On the “I want a pony” side of things, I’d like to capture screenshots at any time during the test. When the test fails because it can’t find an element, I’d like more debugging information, or perhaps the ability to get snapshots of the HTML markup for certain steps. I haven’t seen an option that appends new recorded steps to an existing test.

There’s more to talk about with Ghost Inspector, but I hope this gives you a helpful starting point.

Thanks to Venkata Koppaka for showing me this tool and sorting out the requirements.

Originally published at Falafel Software Blog.