Playwright end to end test automation

Published in

Aeturnum

14 min readMay 23, 2022

1 what is Playwright?

Playwright is a Headless Browser Testing framework designed by Microsoft. This is an open-source NodeJS based framework. Playwright was released for public by Microsoft on 31st January 2020.

Although this is quite new to the IT industry, Playwright has gained the attention of the majority of IT professionals due to the smartness of the framework and because of the fact that Playwright is built by the original creators of Puppeteer, which happens to be the go-to tool for automation when speed and performance of automated tests are the target of an automation project.

2 So much attention during a short time …

Let’s see why Playwright has caught many people’s attention;

Pros of using Playwright for test automation

It is open-source and free.
Setup and configuration is easy.
Provides wider range of coverage for cross browser testing, hence supports, Chrome, Firefox and WebKit.
Supports Windows, Linux as well as macOS.
Supports multiple languages as JavaScript, Java, Python, .NET and C#.
Supports test runner frameworks like Mocha, Jest and Jesmine.
Presents headless browser testing with event-driven architecture.
Can be integrated with major CI/CD tools like Jenkins, CircleCI, Azure Pipeline and TravisCI.
Highly useful for performing cross browser testing for complex applications due to its accuracy, wide coverage and high speed.
Playwright ensures auto wait. Therefore testers don’t have to write code for the wait explicitly because Playwright readies the UI in the backend before test interacts with web elements as soon as the test script is run.
Playwright auto wait feature ensures that the requested action is only performed when relevant checks of the element are completed, therefore accuracy of test results tend to be high.
Supports parallel testing through browser context, thus helps in scenarios where multiple browser tabs have to be opened simultaneously.
Test generation in Playwright lets user perform actions in the browser and it will auto generate the code for the user interactions inside the inspector, thus saving time which takes to write the test code.
Has built in reporters like List, Dot, Line, JSON, JUnit, and HTML. Also can create custom reports.
Has debugging tool support like Playwright Inspector, VSCode Debugger, Browser Developer Tools, and Trace Viewers Console Logs.

Cons of using Playwright for test automation

Playwright is new to the IT industry, therefore has room for improvement always.
Doesn’t support IE11 (This is mostly not considered in modern development).
Doesn’t support native mobile apps but can be used with emulators.
Not much of community support since it’s still growing, only Playwright official documentation is present.

3 Let’s setup Playwright

Get started with Playwright

Step 1: Download Visual Studio Code from the official site https://code.visualstudio.com/

Step 2 : Install Visual Studio Code

Step 3 : Download NodeJs from the official site https://nodejs.org/en/

Step 4 : Install NodeJs

Step 5 : Check if NodeJs is installed properly by going to terminal and typing below command.

Node -v

Should node version if NodeJs is installed properly.

Now let’s install Playwright

Step 1: Go to your project workspace location from files. In my case my workspace is on “/home/tharushi/workspace”.

i. Go to “/home/tharushi/workspace”

ii. Open terminal from that directory

Step 2: Now let’s create a Playwright base project in our workspace directory using the command ‘npm init playwright@latest <project name>’

npm init playwright@latest playwright-demo

iv. Once we hit the command, we are asked to answer few questions as follows;

Q1 : Do you want to use Typescript or Javascript?

A : Let’s select Typescript for the demo here.

Q2 : Where to put your end to end tests?

A : Let’s click ‘Enter’ to submit auto filled ‘tests’ folder.

Q3 : Add a GitHub actions workflow

A : Let’s click ‘Y’ to say yes and wait until everything sets up.

You should be seeing a result like this in terminal.

After the project is created, you will see the finished look as follows;

Step 3: Now navigate to the created project directory from cmd.

cd playwright-demo

Step 4: Now open VS Code from that directory

code .

Now VS Code will be opened.

Step 5: On the left side bar click on ‘Extensions’ icon

Step 6: Enter ‘Install playwright’ on the marketplace search bar.

Step 7: Select the Playwright installation result which has the Microsoft verified icon.

Step 8: Click ‘Install’ button, once installed you will see a preview like below.

Step 9 : Click on ‘file’ icon from left side bar and you should see the base project structure as follows;

Keen to know what our base project auto generated files do?

.github file

This file has GitHub workflow related CI/CD configurations and operating system dependencies on GitHub Actions.

playwright-report

After each test run, the result is saved to this folder as a HTML report.

tests

We write all the test scripts inside this tests folder.

playwright.config.ts

Config file consists of configurations that are needed for us to enjoy features that Playwright has to offer.

Let’s see which configurations are in it;

i. Maximum time one test can run for

ii. Maximum time expected to wait until a condition is met.

iii. Should test files run in parallel.

iv. Which reporter to use (html, json, xml etc.).

v. If trace should be collected during failed tests.

vi. Browser configurations for the project.

vii. Which folder contains images, videos, traces etc.

I have done few customizations to the config file as below. (Read comments to identify added customizations).

What else to know before writing your own test script?

Annotations

Annotations assist in dealing with failures, flakiness, skip, focus and tag tests. Let’s focus on some key annotations;

test.skip(title, testFunction)-When a test is marked with this annotation, it won’t run. Basically that test is skipped when running.

test.fail()-When a test is marked as failing, playwright makes sure the test exactly fails, and in case it passes, playwright will notify us.

test.only(title, testFunction)-To run only few tests among a lot, we can mark the tests that we want to run as focused tests. Then only those marked tests will run.

test.describe(title, function)-A logical name can be given to a set of tests by creating a group.

IMPORTANT : There are a lot more annotations that are used in Playwright. You can click here(annotations) and visit Playwrights official site for more knowledge on annotations.

Assertions

‘expect’ library is used in Playwright for test assertions. Library provides matchers like toEqual, toContain, toMatch and toMatchSnapshot.

Here are some assertions;

IMPORTANT : There are a lot more assertions that are used in Playwright. You can click here(assertions) and visit Playwrights official site for more knowledge on assertions.

Selectors and Locators

Selectors are the strings that are used to create locators.

Locators are the only way to find elements on a page. There are many ways that we can locate elements. They are by xpath, id, class,css selectors, etc.

Here are some locator examples;

CSS selector

Text selector

IMPORTANT : There are a lot more selectors that are used in Playwright. You can click here(selectors), and here for locators to visit Playwrights official site for more knowledge on selectors and locators.

Let’s learn CLI commands needed for our test run

Run all the tests

npx playwright test

Run a single test file

npx playwright test tests/todo-page.spec.ts

Run a set of test files

npx playwright test tests/todo-page/ tests/landing-page/

Run tests in headed browsers

npx playwright test --headed

Run tests in a particular browser

npx playwright test --browser=webkit

Run in debug mode with Playwright Inspector

npx playwright test --debug

Let’s verify running the sample test

So far we know a background about Playwright and have successfully installed Playwright! But do we know if our tests will run? Let’s find out!

The sample test file is auto generated with the creation of our base project. We run this to check if our tests will pass without any issue so we can continue on writing our own tests.

Step 1 : Now click on ‘files’ icon in VS Code, right click on the free space in left side panel and select ‘Open in integrated terminal’

Click ‘Ctrl + ~’

Step 2 : Let’s go to files -> playwright.config.ts file -> comment firefox and webkit to use chromium only.

Then, in the VS Code terminal, type the following command to run our example test file

npx playwright test

It should look like below after a successful execution. You will see how many tests has passed.

Right! Our tests run successfully. Next we’ll set our demo project to POM (Page Object Model) design pattern.

Here we are setting our project according to a design pattern due to the code reusability which helps to prevent small issues that can cause major problems later on. Also using a design patter helps improves code readability for coders.

What is POM (Page Object Model)?

POM is a design pattern which we can store all objects or elements in a class and based on that object, we are able to write the functions.

Non POM structure projects has its elements, methods test methods everything in one class.

Ready? Let’s get started!

Let’s create our sample demo project now

Step 1: As a best practice, first off commit your initial base project to GitHub.

And please continue commit new changes to your repository that point onwards.

If you are a newbie and need to know how to proceed with this step, refer to this article here.

Step 2: In order to do my demo I’m using https://the-internet.herokuapp.com/login site here.

Kudos! 👏 to the original creator of the sample site.

Site reference : https://testersdock.com/text-input-assertion-playwright/

Step 3: Create a folder called ‘data’ : Here we will be having all the mock data needed throughout our demo project in json file format.

Inside ‘data’ folder, let’s now create a json file called ‘login.mock.json’
Inside the created ‘login.mock.json’ file, we’ll add below user credentials, so we can use this data throughout our testing for login purposes.

Step 4: Next we’ll create a folder called ‘utils’ : Here we will be storing the utils needed for the demo project. In real life, this will be where we store environment (dev, qa, prod) URLs of projects.

Inside ‘utils’ folder, create a file called ‘environment.ts’. We’ll add our test environment URL here as follows;

Now we have come to the heart of our topic.

Step 5 : We’ll create PAGES : This is where we create separate classes for each page in the website and we develop functions needed for each page to be used for test scripts.

First off, we’ll create a ‘Login.page.ts’. Now we’ll identify the elements in login page and let’s add our functions.

Now let’s create ‘logout.page.ts’ file under page folder. We will be implementing logout elements and functions inside this file. Code is as below;

Now let’s create ‘common.page.ts’ file under page folder. This is used to store common component data such as alerts. Let’s add following code for our common file.

Step 6 : Then we’ll create a ‘media’ folder : Here we will be storing our screenshots and videos of the tests we run.

We’ll create sub folders under our ‘media’ folder called ‘screenshots’ and also ‘videos’.

Step 7 : We are creating our ‘tests’ folder now : We are using functions and elements created in page classes in test scripts. All expectations, verifications of functionality behaviors are written in test files.

We’ll create a file called ‘login.specs.ts’ inside our ‘tests’ folder.
Here we’ll add our test to login to the system by providing username and password. Then we’ll verify the alert message received after a successful login. Furthermore, we are taking a screenshot here after a successful login.

Let’s create ‘logout.specs.ts’ inside the tests folder. Here we’ll first login to the web site using ‘login.page.ts’ file’s reusable login function.
Then we’ll assert to check if the URL matches our expected result. Then we’ll click logout button and check if landed page URL matches out expected result.
Finally we are checking if we get the alert message for logout successfully. Here’s the code for logout test script;

Step 8: Let’s run our login test now

Open the VS Code integrated terminal and hit the below command to run our login test script;

npx playwright test <test file name> — headed

npx playwright test login.spec.ts --headed

Then we’ll see the browser pops up cause we put ‘ — headed’ which helps in simulating our test scenario in the chromium browser.
The final pass results should be shown in terminal as below;

After the successful execution of the test script, you can view the generated test report as an html page under the given folder location by us in the playwright config file.
If you want to load the report in the browser, you need to hit the following command that is provided in the terminal itself.

npx playwright show-report

You can right click on the generated html report file and select ‘open in browser preview’

Here are all the steps in running our login test in a nutshell

Step 9: Let’s run all our tests now

Open the VS Code integrated terminal and hit the below command to run our login test script;

npx playwright test --headed

The final pass results should be shown in terminal as below;

After the successful execution of test script,s you can view the generated test report as an html page under the given folder location by us in the playwright config file.
If you want to load the report in the browser, you need to hit the following command that is provided in the terminal itself.

npx playwright show-report

You can right click on the generated html report file and select ‘open in browser preview’

AMAZING features of Playwright!

Now that we know how to set up our own Playwrights project, write test cases, run those test scripts and generate reports too, we’ll now move to the amazing facts and features in Playwright.

⚫Trace

Playwright Trace Viewer is a GUI tool that helps exploring recorded Playwright traces after the script run. It opens traces locally in your browser.

Let’s see how we do a trace for our test.

Step 1: We’ll go to our playwright.config.ts file and add ‘on’ value to trace key.

Step 2: Now let’s run our test suite again. As described earlier, hit the report generation command in the integrated terminal, and then you’ll see the generated test report opened in the browser.

Towards the end of the report, you’ll see the trace report as follows;

Step 3: Furthermore, you can click on this trace report, and you can observe network calls, steps, before and after steps, console and time stamps as follows;

⚫Code Generation

If we run codegen and perform actions in the browser, it will automatically write the test code for us with regards to the user actions we perform in the browser, just like a magic!

This magic portion of course saves out time and effort.

Let’s see how we can adapt this into our demo project.

Step 1: First go to the integrated terminal and enter the below command. This will open browser and code inspector for us to perform our user interactions.

We are adding our website URL here, and this should be customized depending on which site you are performing user interactions on.

npx playwright codegen https://the-internet.herokuapp.com/login

Step 2 : Next, perform the user interactions you prefer. I will be login into the website, then logging out of the website and stop the inspector recording.

Step 3: Once we are done with step 2, copy the auto generated code from the test inspector,

Then create a file called ‘codegen.spec.ts’ and paste the copied code there, save it.

Step 4: Then we can run the ‘codegen.spec.ts’ file using the below command and we will see the user interactions we performed in the browser.

npx playwright test codegen.spec.ts --headed

Here are all the steps that we discussed above in a nutshell;

And that’s how we do that magic! 🎉

⚫Debug

Visual Studio Code Debugger helps in inserting break points to our Playwright test code, so we can pause and resume execution of the test scripts, in order to debug our code.

Let’s debug our ‘logout.spec.ts’ file and try out!

Step 1 : Open ‘logout.spec.ts’ file from VS Code

Step 2 : Add a break point for;

expect(page.url()).toBe(“https://the-internet.herokuapp.com/secure");

Step 3 : Click on ‘Testing’ icon from left side panel

Step 4 : Right click on ‘logout.spec.ts’ file and hit ‘Debug test’

Everything up to step 4 should look like this;

Output : Then you will see the code stops after logging in the user and verifying if the landed URL is https://the-internet.herokuapp.com/secure.

Step 5 : From this point onwards, we can run next steps one by one using ‘Step over’ feature provided in VS Code.

Let’s see how we performed out debug test;

So here we come to the end of our article Playwrights end to end automation.

Here’s the git repository of the demo project I implemented https://github.com/desilvawatp/playwright-demo.git. Feel free to make a clone of the project, and play around! 😃

Hope you got an overall idea about this amazing tool Playwright. But this is not it. Playwright offers many more extra ordinary automation features other than what is described above. That’s food for thought!

Let’s hope this amazing tool will soon be the go-to tool for automation! And I will see you soon with a CI/CD integration focus on Playwright.

References : https://playwright.dev/, https://testersdock.com/text-input-assertion-playwright/, https://the-internet.herokuapp.com/login, https://docs.microsoft.com/en-us/microsoft-edge/playwright/