When to use Serenity Ensure Instead of a Questions Class for your Assertions with Screenplay and Serenity BDD

Everyone who has automated using the Screenplay Pattern and the SerenityBDD framework knows that assertions take place on Questions classes that are responsible for validating the status of the elements on the screen to check if a scenario is successful or not. In case the scenario fails, the class will throw an exception (as a good practice, it might be an exception class explaining what happened) to display a better message in the living documentation than a simple generic exception.

In this article, we are going to talk specifically about Questions and how Serenity Ensure improves the assertions and exception handling which make our lives as testers better and easier while decreasing the development time.

Context

The most common way used when trying to check the status of the application, is to create a Question class that returns a Boolean, String, Integer or whatever we need to validate the successfulness of a scenario. If we want to handle a failure or error, we create an Exception class that extends AssertionError and if an unexpected behavior occurs, the living documentation is going to show the message that is relevant to the Exception.

Let’s try to understand what a failure and error are:

╔═════════════════════════════════╬════════════════════════════════╗
║            Failure              ║               Error            ║
╠═════════════════════════════════╬════════════════════════════════╣
║ - Shown in red                  ║ - Shown in orange              ║         
║ - Application with unexpected   ║ - There is an error with the   ║
║   behavior                      ║ test                           ║
║ - Assertion Error               ║ - Driver Exceptions            ║           
╚═════════════════════════════════╩════════════════════════════════╝

So what happens when using Questions with custom exceptions compared using Ensure in the same scenario?

1. Let’s try to create an assertion using a custom Question and Exception class

Question class

Exception (a simple one)

Step Definition using the above classes

After running the test, the living documentation appears as following:

Assume that the LBL_TITLE_AUTHENTICATION locator it’s not well mapped, so the expected result must be an error. Using above implementation, the test case result is a failure, that should be interpreted as an application problem, but the error is in the test case itself, so we’ve not got the expected result.

2. Reproduce the same scenario, but using an assertion using Serenity Ensure

StepDefinition method using Serenity Ensure directly

The result after running the scenario is as follows:

Having in mind the same assumption, what Ensure did for us, was to take control on the assertion and perform a better result. Plus, the amount of code was less and it continues providing a good readability.

But, what if we have a well mapped locator but in an inconsistent or unexpected state? Then, what we should get, is an application error (the red one).

BTN_CONTINUE locator is well mapped, but not enabled

The resulting documentation is as follows:

Using a well mapped locator, Serenity found an element in the page but, in a non expected state. So when you have a problem with the application, the expected result, should be a failure.

In conclusion, to ask for the state of the application, not just Questions can be used. Serenity Ensure gives a wide range of fluent assertions, it’s easy to use and avoids making the test suite architecture more complex. Also, there is no need to create custom exceptions when using Ensure, because it gives better and more understandable feedback messages than Selenium does. Finally, when using questions an extra care must be taken to define the correct message or the correct type of error that will be printed on the living documentation to avoid unexpected results in the living documentation.

Created by Ycuartasj