Shadow DOM in .NET Selenium

What changed in Chromium 96 and what to do about it

Selenium logo in Shadow with .NET

Numerous bugs have been reported for how Chromium v96 has caused working with the Shadow DOM in Selenium to break. This is a feature, not a bug! Let’s break down what changed and how you can manage it in .NET.

Note: versions of this article are also available in: Java, Ruby and Python

As of Chromium v96, shadow root values are compliant with the W3C WebDriver specification, which now includes definitions for getting an element’s shadow root and locating elements in a shadow root. This applies to Chrome, Edge and Opera, and we can expect to see implementations in Firefox and Safari at some point as well.

If you’ve been working with shadow DOM elements in .NET you’ve been doing something like this:

var shadowHost = driver.FindElement(By.Id("shadow_host"));
var js = (IJavaScriptExecutor) driver;
var script = "return arguments[0].shadowRoot";
var shadowRoot = (IWebElement)js.ExecuteScript(script, shadowHost);
var shadowContent = shadowRoot.FindElement(By.Id("shadow_content"));

(see full example on GitHub)

Safari and Chromium 95 and below still support this code in both Selenium 3 and Selenium 4. When you call ExecuteScript(), Selenium looks at the return value and if it determines it includes an IWebElement, it will convert it automatically. Chromium v96 changed the return value so that it isn’t able to be converted to an element. Instead, you get this error (in both Selenium 3 and Selenium 4.0):

System.InvalidCastException: Unable to cast object of type 'System.Collections.Generic.Dictionary`2[System.String,System.Object]' to type 'OpenQA.Selenium.IWebElement'.

Selenium 4.0 didn’t take into account this use case, but when updating to Selenium 4.1, we have a new casting exception:

System.InvalidCastException: Unable to cast object of type 'OpenQA.Selenium.ShadowRoot' to type 'OpenQA.Selenium.IWebElement'.

This error suggests that you should cast it to a ShadowRoot, but actually you want to use the interface that IWebElement inherits from— ISearchContext:

var script = "return arguments[0].shadowRoot";
var shadowRootObj = js.ExecuteScript(script, shadowHost);
var shadowRoot = (ISearchContext) shadowRootObj;

(see full example on GitHub)

Because it is a superclass, it is actually backwards compatible for all versions of Chromium and Safari. If you work with more than just the latest versions of a Chromium browser, this is what you want to use.

If you’re only working with the latest Chromium version, you can take advantage of the new GetShadowRoot() method and avoid ExecuteScript entirely:

var shadowHost = driver.FindElement(By.Id("shadow_host"));
var shadowRoot = shadowHost.GetShadowRoot();
var shadowContent = shadowRoot.FindElement(By.Id("shadow_content"));

(see full example on GitHub)

Notice that this has only discussed Chromium and Safari without mentioning Firefox. Firefox does not currently support getting a shadow root with JavaScript. It does support getting the child elements of a shadow root, though, and it is supported by all versions of Firefox and Selenium. To get the element you’re interested in, you have to iterate over the results:

var shadowHost = driver.FindElement(By.Id("shadow_host"));
var js = (IJavaScriptExecutor)driver;
var script = "return arguments[0].shadowRoot.children"
var childrenObj = js.ExecuteScript(script, shadowHost);
var children = (IEnumerable<IWebElement>)childrenObj;
IWebElement shadowContent = null;
foreach (IWebElement element in children) {
    if (element.GetAttribute("id").Equals("shadow_content")) {
        shadowContent = element;
        break;
    }
}

(see full example on GitHub)

The especially tricky issue here is what to do about Selenium 3. Selenium 3 doesn’t support the new ShadowRoot implementation at all. The most obvious answer is to just upgrade to Selenium 4 already; it’s easier than you think and now is the best time to do it. If you just need to get something working right away and please just show you how to make it work in Selenium 3 already, here is an ugly, hacky solution for you:

var shadowHost = driver.FindElement(By.Id(shadow_host"));
var js = (IJavaScriptExecutor)driver;
var script = "return arguments[0].shadowRoot";
var shadowRoot = js.ExecuteScript(script, shadowHost);
var objDict = (Dictionary<string, object>)shadowRoot;
var id = (string)objDict["shadow-6066-11e4-a52e-4f735466cecf"];
var remoteDriver = (RemoteWebDriver)driver;
var element = new RemoteWebElement(remoteDriver, id);
var shadowContent = element.FindElement(By.Id("shadow_content"));

(see full example on GitHub)

That should be everything you need to get the shadow DOM elements you need in each of the browser versions. Happy testing!

Additional References:
* Language-agnostic version of this article on my website,
* Video with Java examples on YouTube