Shadow DOM in Java Selenium

What changed in Chromium 96 and what to do about it

Selenium logo in Shadow with Java

Lots of people are complaining that Chromium v96 has broken their shadow root code in Selenium. The new implementation is a feature, not a bug! Here’s how to get shadow DOM elements in each browser and version.

Note: versions of this article are also available in: Ruby, Python and .NET

What happened in v96 of Chromium browsers is that shadow root values were made compliant with the updated W3C WebDriver specification. The spec now includes definitions for getting an element’s shadow root and locating elements in a shadow root. This applies to all Chromium browsers (Chrome, Edge, Opera, etc.); and both Firefox and Safari have implementations underway.

The way people are used to accessing shadow DOM elements is with JavaScript. If you are already working with shadow roots in Chrome, Edge or Safari, most likely your code looks like this:

By hostLocator = By.cssSelector("#shadow_host"); WebElement shadowHost = driver.findElement(shadowHostLocator); JavascriptExecutor jsDriver = (JavascriptExecutor) driver; String script = "return arguments[0].shadowRoot" Object shadowRootObj = jsDriver.executeScript(script, shadowHost); WebElement shadowRoot = (WebElement) shadowRootObj; By contentLocator = By.cssSelector("#shadow_content"); WebElement shadowContent = shadowRoot.findElement(contentLocator);

( see full example on GitHub)

That code works for Chromium browsers before v96 and Safari in both Selenium 3 and Selenium 4. Selenium looks at return values from executeScript() commands and if it detects an element, it automatically converts it. With Chromium v96+, the JavaScript returns something that is not identifiable as an element, so Selenium recognizes it as a TransformedEntriesMap. You end up with this error:

java.lang.ClassCastException: class com.google.common.collect.Maps$TransformedEntriesMap cannot be cast to class org.openqa.selenium.WebElement (com.google.common.collect.Maps$TransformedEntriesMap and org.openqa.selenium.WebElement are in unnamed module of loader 'app')

Selenium 4 recognizes the return value and automatically converts it to a ShadowRoot instance. But the above code is still broken! You end up with a different casting error:

java.lang.ClassCastException: class org.openqa.selenium.remote.ShadowRoot cannot be cast to class org.openqa.selenium.WebElement (org.openqa.selenium.remote.ShadowRoot and org.openqa.selenium.WebElement are in unnamed module of loader 'app')

This error suggests that you should cast it to a ShadowRoot, but not so fast. Selenium doesn’t want you to use the ShadowRoot class directly. Instead you should cast this to theSearchContext interface.

String script = "return arguments[0].shadowRoot" Object shadowRootObj = jsDriver.executeScript(script, shadowHost); SearchContext shadowRoot = (SearchContext) shadowRootObj;

( see full example on GitHub)

The advantage of this approach is that it works for all versions of Chrome, Edge and Safari. If you need to work with multiple versions these browsers, this is still your best option.

If, on the other hand, you’re only interested in working with the latest Chromium version, there’s a better way! Use the brand new getShadowRoot() method- no need for a JavaScriptExecutor.

By hostLocator = By.cssSelector("#shadow_host"); WebElement shadowHost = driver.findElement(shadowHostLocator); SearchContext shadowRoot = shadowHost.getShadowRoot(); By contentLocator = By.cssSelector("#shadow_content"); WebElement shadowContent = shadowRoot.findElement(contentLocator);

( see full example on GitHub)

What about Firefox? Firefox is special. Until it implements the specification, you have to use different JavaScript than for Chromium or Safari. It works for all versions of Firefox and Selenium and it looks like this:

By hostLocator = By.cssSelector("#shadow_host"); WebElement shadowHost = driver.findElement(shadowHostLocator); JavascriptExecutor jsDriver = (JavascriptExecutor) driver; String script = "return arguments[0].shadowRoot.children"; Object childrenObj = jsDriver.executeScript(script, shadowHost); List<WebElement> children = (List<WebElement>) childrenObj; WebElement shadowContent = null; for (WebElement element : children) { if (element.getAttribute("id").equals("shadow_content")) { shadowContent = element; break; } }

( see full example on GitHub)

Finally, the section you’ve all Googled for- what if you to upgrade to Selenium 4? What do you do? First, seriously, just upgrade to Selenium 4 already- it’s important; it’ll save you headaches in the future if you do it now. But fear not, you stubborn testers, if you need to access elements in the Shadow DOM you have this hacky code available to you in Selenium 3:

By hostLocator = By.cssSelector("#shadow_host"); WebElement shadowHost = driver.findElement(shadowHostLocator); JavascriptExecutor jsDriver = (JavascriptExecutor) driver; String script = "return arguments[0].shadowRoot" Object shadowRootObj = jsDriver.executeScript(script, shadowHost); Map<String, String> map = (Map<String, String>) shadowRoot; String id = map.get("shadow-6066-11e4-a52e-4f735466cecf"); RemoteWebElement shadowRootElement = new RemoteWebElement(); shadowRootElement.setParent((RemoteWebDriver) driver); shadowRootElement.setId(id); By contentLocator = By.cssSelector("#shadow_content"); WebElement shadowContent = shadowRoot.findElement(contentLocator);

( see full example on GitHub)

Now you should have a good idea of what changed and why, and what all solutions you have available to you in the different browser and Selenium versions. Happy testing!

Additional References:
* Language-agnostic version of this article on my website,
* Video with Java examples on YouTube

Originally published at https://www.linkedin.com.