CodeX
Published in

CodeX

The Testing Library meets Selenium

The Testing Library enables testing as a user. It’s available for multiple JavaScript frameworks (e.g. React, Vue, Cypress). I realized it was possible and valuable to bring it to Kotlin’s Selenium.

What

I developed a set of custom Selenium locators (e.g. ByRole, ByText) that wrap the corresponding Testing Library queries (specifically, the DOM Testing Library):

// with the Selenium Testing Library (Kotlin)
driver.findElement(ByAltText("First name"))
driver.findElement(
ByRole("heading", name = "/as a user/i".asJsRegex())
)
// with the Testing Library (JavaScript)
screen.queryByAltText('First name')
screen.queryByRole('heading', { name: /as a user/i })

I believe the best examples are given by the tests, so check them out. As a sneak peek, here are some more examples of the Testing Library’s Core API:

I also implemented the Testing Library User Interactions API:

I’ve packed all this into the Selenium Testing Library (for Kotlin, but it also works in Java) and open-sourced it (by the way, someone did the same for Python).

Why

There was more than one reason to create this library:

  • Contribute to the open source world, hoping to have created something useful for others;
  • Dive deeper into Selenium, the Testing Library, and webpack;
  • Learn how to publish a library to Maven Central (and perhaps how to automate it with GitHub actions);
  • Use Selenium the way I believe UI testing should be done — testing as a user.

Why do I believe that testing as a user is superior to relying on technical aspects (ids, classes, tag names, test ids, XPath/CSS selectors)? It’s related to the goals of automated testing:

  • The better the tests resemble the ways the users interact, the more confidence I get from them (note that this doesn’t imply end-to-end testing; the Testing Library can be used to test simply React components). I feel safer and more guided when refactoring if I’m able to change the underlying implementation details. CSS and XPath selectors and the HTML structure itself are all volatile low-level details. Test ids are a workaround, but that means creating implementation code solely for the sake of testing (a testing antipattern).
  • If you, as a developer, have trouble finding UI elements, how can the users? When testing as a user, you’re compelled and encouraged to follow good usability practices like writing semantic HTML (e.g. article, section, nav) and providing accessibility labels (e.g. title, alt, aria-label).
  • I get much more documentation power from the tests when they describe real-life usage scenarios. If I read those tests a few months later, I can easily relate to their goals and I don’t get lost in technical details.

Why not just use the Testing Library (in JavaScript or TypeScript) in the first place? I did it in some past projects (and that’s how I knew the Testing Library), but not everyone is working in those languages/runtimes. Some real-world projects are already running and using Selenium for JVM, and they also deserve the benefits of the Testing Library. Also, I appreciate what a compiled language can bring to a test suite.

Without this library, you’d have to resort to ids, classes, and test ids. If you didn’t want to depend on those technicalities, you’d need lots of utilities. In some cases, it wouldn't even be possible. For example, a case-insensitive-contains search could not be achieved with XPath (at least with current browsers).

Beware that you still need to wait for the elements if they’re not immediately available (as with native locators); you can resort to Selenium’s waiting capabilities. However, if you prefer, you can rely on the Testing Library’s waits with an API that I added as an alternative:

val firstName = driver.queryBy(
PlaceholderText, "first name", mapOf("exact" to false)
)

How

How have I integrated the Testing Library in Selenium? By evaluating its JavaScript so it’s available to be used later (equivalent to pasting and running it in the JavaScript console). Then, I can invoke the Testing Library functions and map the results back to Java/Kotlin objects. Here’s the juicy part of the technique:

// 1️⃣ in JS, store a handle to TL's screen:
import {screen} from '@testing-library/dom'
window.screen = screen
# 2️⃣ generate the TL script to be injected:
webpack --mode production
cp dist/testing-library.js ../lib/src/main/resources
// 3️⃣ evaluate the TL script so it gets stored in screen:
driver.executeScript(
{}.getResource("/testing-library.js").readText()
)
// 4️⃣ call a TL function from screen and rely on Selenium to map the results to WebElements
val results = driver.executeScript(
"return screen.queryAllByRole(arguments[0])",
"listbox"
)

As you can see, all the heavy lifting is done by webpack, the Testing Library, and Selenium. But there’s a missing piece: how to make these queries transparent to Selenium? I’d want to use them such as the native ones (e.g. driver.findElement(By.Id("profile"))). Selenium is easily expandable with new locators: inherit from By and implement findElements. Here’s an example of a custom locator (not from my library):

I can use that custom locator like any native Selenium locator: driver.findElement(ByAttribute("href", "http://example.com")).

Get, Query, or Find? One or All?

The Testing Library Core API has three families of queries: get* , query* , find*. Which one to pick? get* fails when elements are not found and I wanted to leave that for Selenium. find* waits until the elements are available, and again, Selenium has waiting mechanisms. That leaves query* as the more flexible option and the one I used.

Each family of functions has a singular version (e.g. query) and plural version (e.g. queryAll). I used the plural because to create Selenium locators, we only need to implement findElements (plural). Selenium provides findElement (singular) based on it.

To port or to adapt?

Injecting the Testing Library script and bridging it to Kotlin/Java is known as the adapter pattern. However, my initial approach was to rely on Selenium locators (e.g. xpath, cssSelector) to port the Testing Library behavior aiming to achieve feature parity. It turned out this was a limited approach because:

  • Selenium locators can’t compete with Testing Library search functions due to WebdriverIO and browser limitations (e.g. a case-insensitive text search by XPath);
  • I’d have to port all functionality, which is a lot of work;
  • I’d likely introduce bugs;
  • I wouldn’t get library updates easily.

I ended up using the injection approach, which is valid based on the MIT license of the Testing Library.

📝 If you’re curious about the old approach, check the project’s Git history.

Won’t it be slow?

Injecting the Testing Library has a performance impact. However, consider the following optimizations:

  • The source JS scripts are minified (each is around 250KB).
  • The JS files are read at most once from the filesystem after which they’re read from memory.
  • Each script is independently injected at most once (until there’s a page navigation or refresh) and only if/when needed.

The injection times roughly vary between 20 and 40 milliseconds. Considering the benefits of the Testing Library, I believe it pays off.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Luís Soares

Luís Soares

516 Followers

I write about automated testing, Lean, TDD, CI/CD, trunk-based dev., user-centric dev, DDD, coding good practices,