A Guide to Working with Shadow DOM using Selenium
Recently at Rate, we have decided to implement Shadow DOM rendering with one of our products, RateX, which is a browser extension that allows users to make cross-border payments with the lowest exchange rates. Additionally, it helps users to apply the latest coupon codes, thus saving more money for our users.
Thus, our extension needs to interact with many e-commerce sites. Before implementing Shadow DOM, we would get issues where the sites’ CSS interact with our extension’s, causing conflicts and interfering with how our extension is rendered. Now that we have started using Shadow DOM in our extension, issues like this do not crop up anymore. In case you want to find out more about how we use Shadow DOM, Rate Engineering will be publishing an article on just this topic in the coming weeks.
The problem
However, in using Shadow DOM, another problem has cropped up, specifically in how we implement automated testing for our browser. We use Selenium, one of the most popular automation testing tools for web applications.
The issue is that Selenium does not provide explicit support for working with Shadow DOM elements. For example, when we attempt to locate a Shadow DOM element on the WebDriver instance using the findElement function, an exception will be thrown saying the element cannot be found. Thankfully, there are easy ways to work around this.
// This does not work!
await driver.findElement(By.css(CSS_SHADOW_DOM_ELEMENT));Locating Shadow DOM elements
Let’s continue using our example of locating a Shadow DOM element. The first thing we need to do is to locate the shadow host, the regular node that the Shadow DOM is attached to.
This is simple enough to do, as this is merely a matter of using the findElement() function to locate the element on the WebDriver instance itself. Using the executeScript() function, we then execute a snippet of Javascript that will allow us to retrieve the shadow root using the shadow host from before. Below is an example of how this might be coded out in Javascript:
async function getExtShadowRoot() { let shadowHost; await (shadowHost = driver.findElement(By.css(CSS_SHADOW_HOST))); return driver.executeScript("return arguments[0].shadowRoot", shadowHost);}
Now that we have our shadow root, we can then use that to find the Shadow DOM element we need. We do this by using the findElement() function, but instead of locating on the WebDriver instance as before, we locate it on the shadow root WebElement.
async function findShadowDomElement(shadowDomElement) { let shadowRoot; let element; await (shadowRoot = getExtShadowRoot()); await shadowRoot.then(async (result) => { await (element = result.findElement(By.css(shadowDomElement))); });
return element;}
And that’s it. Using these two functions, we can locate the Shadow DOM element we need.
Explicit waits for Shadow DOM elements
But what if we want to use Explicit Waits to wait for a Shadow DOM element to be located/visible/enabled? Selenium already provides convenience functions that wait for these conditions to be fulfilled, used in conjunction with the wait() function of a WebDriver instance. Looking at the implementation of the Javascript binding for Selenium, most of these convenience functions will work for Shadow DOM elements the same way as light DOM elements; just pass in the element you obtained through the functions above.
However, there are a few convenience functions that we will need to write our own implementation for, mainly the elementLocated and elementsLocated functions. This is because these functions use the findElements() function on a WebDriver instance, which as we know, do not work with Shadow DOM elements. Below is an example of how you can implement a function that waits for a Shadow DOM element to be located:
async function waitUntilShadowDomElementLocated(element) { let shadowRoot; await (shadowRoot = getExtShadowRoot()); await shadowRoot.then(async (result) => { await driver.wait(() => result.findElements(By.css(element)) .then(elements => { if(elements.length === 0){ return false; // element not found } else { return elements[0]; } }), 5000); });}
Take note that the above code is equivalent to the below code for light DOM elements.
async function waitUntilElementLocated(element) { await driver.wait(until.elementLocated(By.css(element)), 5000);}
Don’t overlook Waits!
While we are on the topic of Waits, I want to digress a little bit and emphasize their importance in browser testing. Quite often, the reason you get a NoSuchElementException is that the browser has not loaded the element yet but the program has already finished looking for it. This is applicable to not just Shadow DOM elements, but also light DOM elements. However, with the introduction of Shadow DOM elements, I first made the mistake of thinking that the NoSuchElementException was thrown because there was a problem in getting the shadow root. It took me a while before I realized that I needed to implement a wait so that there is time for the element to load before the program attempts to locate the element.
Therefore, my advice is to only use the findElement() function only when you know for sure that the element would have loaded by the time the function is called.
Conclusion
Maybe Selenium will eventually provide support for Shadow DOM in the coming future. In the meantime, I hope this guide helps you avoid the headaches I had working with Selenium and Shadow DOM.
References
Webdriver and the Selenium Server: https://www.seleniumhq.org/docs/03_webdriver.jsp#webdriver-and-the-selenium-server
Working with Shadow DOM Elements using Webdriver: http://www.seleniumeasy.com/selenium-tutorials/accessing-shadow-dom-elements-with-webdriver
Using Shadow DOM: https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_shadow_DOM
How to implement Wait for Shadow DOM element to be located: https://stackoverflow.com/questions/49052228/can-i-wait-for-a-subelement-appear-in-selenium-webdriver
Selenium-Webdriver API for Javascript: http://seleniumhq.github.io/selenium/docs/api/javascript/index.html
