Playwright with .NET. What? Is it useful?

Greetings, my fellow coding nerds,
Today I decided to try to research and write articles in parallel. So this is a kind of lab experiment notes.

First of all, YES, Playwright can work not only with JavaScript/TypeScript languages. According to the official documentation at the moment Playwright also supports Python, Java, .NET(C#) languages except the basic ones. And you may already find some articles about usage of Playwright with other programming languages

More than half a year ago I’ve already tried it with .NET. And my first impression was why should I do it even with my primary language when I can easily write it with basic JavaScript or TypeScript. Anyway, sometimes you want to combine it with already existing technologic stack and some preconditions may require a big afford to start from the beginning. Like some microservice tier implementations. So, I decided to give a second chance and try to understand if combination of Playwright and .NET could be useful. So please sit belt and let’s drive

Installation

I’m not going to make something up and go straight to the documentation. Because I think Playwright has one of the best documentations.

My first step will be to create a project with the tests:

dotnet new nunit -n PlaywrightTestsWithDotNet

Next we switch to our newly created folder with a project:

cd PlaywrightTestsWithDotNet

And we need to install “Microsoft.Playwright.NUnit”.

Note: There is an option to install it with MsTest. I don’t know if somebody still use it. Please tell me if I’m wrong.

dotnet add package Microsoft.Playwright.NUnit

Now, let’s build our solution.

dotnet build

Looks like everything is going according to the plan. Actually according to the documentation:)

Sorry, I like this old movie called “Beverly Hills Cop”. At least I did when I was a kid :)

The next step is to install Playwright browsers. Potentially, we can run our tests against local or contenerized browsers. But I want to follow all required steps, so I don’t want to skip this step. As a first step, I will run it against these browsers. To do this, we need to run the following command. Here are the important things

• You need to have PowerShell. IF you don’t have, you can install it
• Please update “netX” to actual version of .NET. I just realized in my case it is already “net7.0”

pwsh bin/Debug/netX/playwright.ps1 install

Hm. It was quick. Now we are ready to go coding. Let’s start with simple test without any Design Patterns. Step-by-step

Sample test

I use Visual Studio to open the project. You can use other editor if you want. E.g. Rider, Visual Studio Code.

I’m inside. There are two files.

Let’s rename our UnitTest1.cs class to something more specific and add a sample test that opens a page and verifies that we are on that page. For testing purposes, I’ll use my test website, which I prefer to use for UI/e2e testing practices.

This is my very simple first test scenario.

• I want to navigate a login page
• I want to verify Header Title and also verify if Login button was attached to the page.

Test should be inherited from base “PageTest” class that provides you access to Page. Object you use to communicate with a page. Also it provides Expect method to perform assertions.

You can run tests using command line. As you may see command also includes parallelization level.

dotnet test — NUnit.NumberOfTestWorkers=5

Or you can use Test Explorer in order to run the tests

By default Playwright runs the tests in Headless mode. So you will not see how it opens browser and do something. In this case, let’s jump into the project configuration

RunSettings

Let’s create a RunSettings file specifically for chromium browser and disabled Headless mode. Also I will include a little bit of SlowMo for debugging.

Use command line or choose runsettings file and run from test explorer

dotnet test — settings:chromium.runsettings

Works. Looks cool.

Now I’m interested in finding more Playwright configuration options. Here is what I found at the moment: Options. I won’t use all of them. But let’s try a few more interesting ones

So I have added a few new settings like

• “BaseUrl” with “TestRunParameters” node
• Number of Test Workers inside of “NUnit” node
• Some recommended settings for debug mode
• Also extended Playwright specific settings for default wait timeout and browser channel.

Also if we already decided to use BaseUrl from the runsettings file, I want to update our test in order to read it from the condiguration file.

Test will still pass. To be honest, I have not found a complete description of the supported Playwright settings for .NET. And this is something that really bothers me. Maybe there is a luck of documentation or maybe the .NET version of Playwright has limited support of config (compare to Node.js).

Honestly I was expecting something similar like perform screenshot with “only-on-failure” options or similar logic with video. But it looks like I need to resolve it on code level.

Above example from Node.js. The same section on .NET will show page is not available. Sad.((

Page Object

Let’s not give up. You think I’d let you do this on your own? Without the most important UI Automation Design Pattern? No way:)

Scenario

The scenario will be a very simple

• Login to the website using UI (It is possible to do through LocalStorage. But not this time)
• Navigate to the Check-boxes page
• Click some check-box
• Verify it was marked

Page Objects implementation

For this scenario we basically need two pages: LoginPage and Checkboxes page to manage the pages.
But I always like to have a very simple class object. This is very useful if we want to add some standard logic or logger in the future. So, in this case, let’s create a very simple “BasePage”.

• In the initial state, I have added protected Page and Assert properties that will definitely be used inside each Page object.
• Also, in my case, I can use the navigation menu from any of the pages. So it is logical, I will add it to the base class and it can be called from any child page object.

Now we can start implementing two other pages that have “BasePage” as a base class.

LoginPage:

This page will contains Locators and methods to perform login operation.

Also I decided to add “ITestPage” interface as a contact. In this case, it require to implement “WaitUntilPageIsLoaded” method. Methods looks pretty the same as in JavaScript implementation. I have even copied locators from one of my previous projects. Good sign locators work the same.

CheckBoxesPage:

This page is also an adapted copy of the above mentioned project. Unfortunately, this page is larger than the previous one. So I split it into three screenshots.

Let’s jump into the Test itself

Tests using Page Object

• Next step was to create a Test class
• Initialize PageObjects
• Call methods
• Perform Assertions

Reminder methods should ASYNC.

Here I have faced with first limitations.

• “Expect” itself is a hard assertion. This means that it will fail as soon as the first assertion fails. Playwright/Test with JS/TS includes soft assert extension. C# version doesn’t.

• Second, parametrized tests fails to run because it does not Isolate Page, but shares it between the tests. So, this one will not work. At least without Spike. I tried to create some Page Stack, but decided to Postpone it.

• Assertion methods are limited. At this moment it works with Locator, Page and Response (API Response) and does not work with specific types.

As an example, in TypeScript, I was returned Boolean value and verified it.

Here I just cannot do it. Sad.

In general, I spent more time to understanding how to create the tests than it was with TypeScript.

Test Hook

Test hooks looks pretty simple and also limited. At this moment it supports only two standard NUnit attributes

[SetUp] attribute — To run before each test

[TearDown] attribute — To run after each test

Yes, Page is not available if you try to use “OneTimeSetUp” or “OneTimeTearDown”

I decided to create a simple examples.

• In setup method, I’m logging in to application
• In teardown method, I’m making screenshot. Logically is to make it only on fail. But this is just an example to confirm it works

One of these actions a little bit shortened a test beginning

Nothing to add more.

API tests

The .NET version of Playwright supports API calls out of the box, just like other programming languages.

For API testing purposes, I’ll use the Sample Countries microservice

I want to search country by its full name. And I’m going to use the V2 endpoint version. Because it contains fewer fields in the response.

Let’s first create a C# model for the response.

In this case I decided to add only few of the fields. Just for testing purposes.

First problem. I have already tried and default JSON parser does not work with “newtonsoft json”. So I created properties in lower case, like original JSON response fields.

Now, let’s create a test.

• This time test should be inherited from the “PlaywrightTest” base class.
• Next, I initialized an API request content with BaseUrl inside the SetUp hook.
• The test itself looks quite familiar. We perform a request. Verify that the status code is OK(200). Then we parse JSON to our model and verify that correct field values are returned.

Of course, this is an example of a very simple request. Anyway, other method calls look pretty much the same as in JS/TS. So, I don’t want to stop here.

Command line filters

I think, it is time to finish. Basic parts were already uncovered. Let’s just quickly consider command line options.

• We have already familiar how to run the tests using runsettings files

dotnet test — settings:chromium.runsettings
dotnet test — settings:firefox.runsettings
dotnet test — settings:webkit.runsettings

• There is also one useful command line flag called “filter”

There are not a lot of options here. We can filter by class names, by test method names and run with specific browsers.

Works as expected

To Summarize

Pluses:

• Auto-waiters work well. You don’t have to create special logic for them. Compare with WebDriver.
• Supports custom locators. More flexible than WebDriver
• Supports API requests. No need for external libraries
• Screenshot, Video and Traces also work out of the box. You can easily enable them.
• Tests are isolated. This means that each test has it’s own local storage, session storage, cookies etc. Playwright achieves this using BrowserContexts which are equivalent to incognito-like profiles.

Minuses

• All the benefits and more available in the original Node.JS version of Playwright
• A lot of logic is still cut. Looks like demo version compared to node version
• May be incompatible with other .NET libraries. In Node.js, it is much easier to integrate with other libraries.
• Does not support the TestCase attribute out of the box. At least page is not isolated in this case. May cause creation of spike solution
• Playwright is designed to simplify the task of test automation. In some complex case with C#, you need to perform additional actions
• Works with a limited browsers (The same as Node version)

In total. My goal was to try and compare Playwright .NET version with Playwright Node.js version. And start was a fresh. But as I said before, unfortunately the .NET version looks like a good demo version. Still a lot of things you need to solve. In the NODE.js version, when I got stuck, I just went to the documentation and found a solution. In case of .NET it might not exist. Integration with other libraries is also limited.

Even if I like C#, personally I wouldn’t use Playwight .NET version at this moment (24.09.2023). It is easy to switch on TypeScript and create tests using Node.js version. Once more, this is only my subjective opinion. Please tell me if you no something I don’t know. BTW: Project available by this link: PlaywrightTestsWithDotNet

Thank you very much for reading this article. I spent a lot of my weekend time researching and understanding it, I’m not going to use it:) Anyway,
“This is the way!”

Have a good day!