The Testing Library meets Selenium

The Testing Library enables testing as a user. It’s available for multiple JavaScript frameworks (e.g., React, Vue, Cypress). I realized it was possible and valuable to bring it to Kotlin’s Selenium.

The more your tests resemble the way your software is used, the more confidence they can give you. (Kent C. Dodds)

What

I developed a set of custom Selenium locators (e.g. ByRole, ByText) that wrap the corresponding Testing Library queries (specifically, the DOM Testing Library).

🔗 GitHub — Selenium Testing Library

Here’s an illustrative example:

// with the library I created (Kotlin+Selenium):
driver.findElement(altText("First name"))
driver.findElement(role(Heading, name = "payments", exact = false))

// with the Testing Library (JavaScript):
screen.queryByAltText('First name')
screen.queryByRole('heading', { name: 'payments', exact: false })

Tests provide the best examples, so check them out.

Besides the Core API, the user-event, fireEvent, and jest-dom are also available. I’ve packed all this into the Selenium Testing Library (for Kotlin and Java) and open-sourced it.

Why

There was more than one reason to create this library:

• Contribute to the open source world, hoping to create something valuable for others;
• Dive deeper into Selenium, the Testing Library, and webpack;
• Learn how to publish a library to Maven Central (and perhaps how to automate it with GitHub actions);
• I wanted to enable Selenium to test web UIs as a user — how I believe UI testing should be done.

Why do I believe that testing as a user is superior to relying on technical aspects (ids, classes, tag names, test ids, XPath/CSS selectors)? It’s related to the goals of automated testing:

• The better the tests resemble how the users interact, the more confidence I get from them (note that this doesn’t imply end-to-end testing since the Testing Library can also be used to test React components). I feel safer and more guided when refactoring the underlying implementation details. CSS, XPath selectors, and the HTML structure are volatile low-level details. Test ids are a workaround, but that means creating implementation code solely for testing purposes (a testing antipattern).
• If you, as a developer, have trouble finding UI elements, how can the users? When testing as a user, you’re compelled and encouraged to follow good usability practices like writing semantic HTML (e.g. article, section, nav) and providing accessibility labels (e.g. title, alt, aria-label).
• I get much more documentation power from the tests when they describe real-life usage scenarios. If I read those tests a few months later, I can easily relate to their goals without getting entangled in technical details.

Why not just use the Testing Library (in JavaScript or TypeScript) in the first place? I did it in some past projects (and that’s how I knew the Testing Library), but not everyone works in those languages/runtimes. Some real-world projects are already running and using Selenium for JVM, and they also deserve the benefits of the Testing Library. Also, I appreciate what a compiled language can bring to a test suite.

Without this library, you’d have to resort to ids, classes, and test ids. You'd need lots of utilities if you didn’t want to depend on those technicalities. In some cases, it wouldn't even be possible. For example, a case-insensitive-contains search could not be achieved with XPath (at least with current browsers).

<input placeholder='Username' />

// 🟠 With pure Selenium
val result = driver.findElements(By.cssSelector("*"))
    .first() {
        it.getAttribute("placeholder")?
          .contains("USER", ignoreCase = true) == true
    }

// 🟢 With Selenium Testing Library
val result = driver.findElement(
   ByPlaceholderText("USER", exact = false)
)

Beware that you still need to wait for the elements if they’re not immediately available (as with native locators) — you can resort to Selenium’s waiting capabilities.

How

How have I integrated the Testing Library in Selenium? By evaluating its JavaScript so it’s available to be used later (equivalent to pasting and running it in the JavaScript console). Then, I can invoke the Testing Library functions and map the results back to Java/Kotlin objects. Here’s the juicy part of the technique:

// 1️⃣ in JS, store a handle to TL's screen:
import {screen} from '@testing-library/dom'
window.screen = screen

# 2️⃣ generate the TL script to be injected:
webpack --mode production
cp dist/testing-library.js ../lib/src/main/resources

// 3️⃣ evaluate the TL script so it gets stored in screen:
driver.executeScript(
   {}.getResource("/testing-library.js").readText()
)

// 4️⃣ call a TL function from screen and rely on Selenium to map the results to WebElements
val results = driver.executeScript(
   "return screen.queryAllByRole(arguments[0])", 
   "listbox"
)

As you can see, all the heavy lifting is done by webpack, the Testing Library, and Selenium. But there’s a missing piece: how do we make these queries transparent to Selenium? I’d want to use them such as the native ones (e.g. driver.findElement(By.Id("profile"))). Selenium is easily expandable with new locators: inherit from By and implement findElements. Here’s an example of a custom locator (not from my library):

data class ByAttribute(private val name: String, private val value: String) : By() {
    override fun findElements(context: SearchContext): List<WebElement> =
        context.findElements(
            cssSelector("[$name='${value.replace("'", "\\''")}']")
        )
}

I can use that custom locator like any native Selenium locator: driver.findElement(ByAttribute("href", "http://example.com")).

Get, Query, or Find? One or All?

The Testing Library Core API has three families of queries: get* , query* , find*. Which one to pick? get* fails when elements are not found and I wanted to leave that for Selenium. find* waits until the elements are available. However, Selenium has wait mechanisms. That leaves query* as the more flexible option and the one I used.

Each family of functions has a singular version (e.g. query) and plural version (e.g. queryAll). I used the plural because we only need to implement findElements to create Selenium locators. Selenium provides findElement (singular) based on it.

To port or to adapt?

Injecting the Testing Library script and bridging it to Kotlin/Java is known as the adapter pattern. However, my initial approach was to rely on Selenium locators (e.g. xpath, cssSelector) to port the Testing Library behavior aiming to achieve feature parity. It turned out this was a limited approach because:

• Selenium locators can’t compete with Testing Library search functions due to WebdriverIO and browser limitations (e.g., a case-insensitive text search by XPath);
• I’d have to port all functionality, which is a lot of work;
• I’d likely introduce bugs;
• I wouldn’t get library updates easily.

I ended up using the injection approach, which is valid based on the MIT license of the Testing Library. An exception to this was that I ported jest-dom rather than bridging it. To bridge it, I had to inject jest and jest-dom, which proved troublesome.

Won’t it be slow?

Injecting the Testing Library has a performance impact. However, consider the following optimizations:

• The source JS scripts are minified (each is around 250KB).
• The JS files are read at most once from the filesystem, after which they’re read from memory.
• Each script is independently injected at most once (until there’s a page navigation or refresh) and only if/when needed.

The injection times vary between 20 and 40 milliseconds. Considering the benefits of the Testing Library, it pays off.

🔗 GitHub — Selenium Testing Library

🔗 The Testing Library at Maven Central