Playwright vs Cypress

HeleneKassandra
Nov 6 · 6 min read
Image for post
Image for post

Not too long ago, Cypress seemed to be the most exciting new end-to-end testing framework out there, quickly growing in popularity within different development teams. However now there’s a new kid on the block named Playwright, and it aims to solve a similar issue — helping developers automate their user-flows in a more user-friendly way.

This article will compare the two, and hopefully make it clearer which testing framework suits your needs the most, by making you aware of their similarities, differences, strengths and weaknesses.

About the frameworks

For an introduction to the fundamentals of Cypress.io, check out my other article named “Testing with Cypress”. In short, Cypress is a tool for setting up, writing, running and debugging tests. It compiles all the tests into Javascript, runs in an instance of a chromium-based browser that includes Chrome, Edge, Brave and Electron. Cypress also just recently added full-support for Firefox browsers.

Playwright is essentially a browser automation tool and the processor of the node library Puppeteer, as it has the same functionality along with several improvements such as Cross-browser testing and device emulations. Playwright is also written and maintained by the same people who created Puppeteer, and are now working at Microsoft. This article won’t cover Puppeteer, but it’s handy to know of its similarities, especially if you’re already familiar with it.

Similarities

Similarly to Cypress, Playwright is an open-source, Javascript-based library, for automating your end-to-end tests. Both aim to provide a single API that developers and testers can use to interact with web applications across the major browser engines. There is a difference between the two when it comes to browser support, but both offer the ability to run tests and interactions in Firefox and Chromium browsers. Other similarities include functionality, like taking screenshots, stubbing requests, and testing on various screen sizes. If you want to run your tests as part of your continuous integration flow, or prefer to run the tests without a UI interface, then you’ll be happy to hear both offer headless options so that you can run your tests directly in the Terminal.

To summarize:

  • Both are Open-source and Javascript-based
  • Can install both as an npm-package.
  • Single API for testing in several browsers (Both support Firefox and Chromium)
  • Share a lot of the same functionality, like screenshots, stubbing and setting custom view-ports.
  • Neither supports testing in IE11
  • Can be run from the Terminal
  • Frequently updated GitHub Repos

Differences

While they both aim to solve a similar issue, they have different ways of doing so. Cypress is more of a “full-package”, and it creates a folder structure along with example files, and you are stuck with the test runner you get with the framework. Playwright, on the other hand, does not make any files and can be configured to work with the test runner of your choice. The two frameworks also run their tests differently. Cypress runs the tests in run-time, and Playwright is promise-based and can run several different browsers and different user contexts in the same test, while Cypress needs to be re-run with the other browser options.

Cypress has gone for a syntax more similar to JQuery, but instead of “$”, it uses the keyword “cy”, and a function name. Having one keyword to access everything, might make more sense for designers and junior front-end developers, less familiar with async and creating instances of objects, than the more Javascript approach Playwright has. The example below shows the syntax differences between the two frameworks, and the test scenario is to check if an element with the class name “App-logo” exists.

As previously mentioned Playwright has a syntax closer to Javascript, where you create instances of objects. The ability to create object instances allows us to run multiple tabs, browsers and user contexts at the same time. The example above shows how Playwright uses the async functionality to wait for a UI-element to appear before continuing the test, Cypress, however, solves a similar issue by automatically re-trying the assertions until it reaches the set timeout.

One of the most significant benefits of Playwright is its ability to test across multiple pages and domains. Along with setting multiple user contexts. Both are very useful if you’re using third-party sign-ins, pop-ups, iframes (such as BankID in Norway) etc. in your application. Cypress, on the other hand, would require you to write separate tests to simulate the different user scenarios and would require you to stub a lot of the requests to work.

To summarize:

  • Playwright works on Webkit-browsers, Cypress does not.
  • You can choose test-runner in Playwright
  • Playwright lets you test in several browsers at the same time.
  • Playwright supports multi-tabs and frames.
  • Cypress doesn’t run in headless mode by default, Playwright does.
  • Playwright awaits UI-elements before running interactions, Cypress re-try assertions until timeout.

Size and performance

When it comes to size and performance, it’s a bit of a mixed bag. Since Cypress has a built-in test runner, Jest has been added to the comparison, as it’s the most popular Javascript test runner, and needed to achieve similar functionality to Cypress in Playwright. Looking at the minified size, Cypress is technically smaller with it’s 1.6MB against the 2.85MB of Playwright + Jest, but where Playwright + Jest shine is when you look at the dependencies where Playwright + Jest has 14, compared to the 125(!) of Cypress.

To test performance, a colleague and I wrote a test in both Playwright + Jest and Cypress. The test scenario covers the following steps:

  1. Go to sparebank1.no
  2. Press “Godta alle” button for the cookie pop-up
  3. Find postal code input and enter 3324
  4. Check if a button with the class “ffe-shortcut-button” and text “SpareBank 1 Modum” is visible.
  5. Click the button, and check if the page now is Sparebank 1 Modum.

The results show that it’s only milliseconds separating the two in terms of speed. Cypress ran the test in 3 seconds, whilst Playwright slightly beat that by completing the test in 2.33 seconds. Essentially it means that both run the test fast, and whilst Playwright was somewhat quicker, it’s not that big of a difference that it should affect your choice of framework.

What to pick?

So which of these frameworks should you choose? The answer is it depends on how experienced you are with testing, and what functionality you find essential. If you’re new to testing and want a more plug-and-play approach that includes everything you need to get started, then Cypress is the best choice for you. It has good documentation and a broader community that makes it easier to get help and find answers to specific scenarios you find challenging.

If you are more familiar with testing, need to test Webkit browsers or your tests need to cover scenarios spanning across multiple pages and domains, then Playwright is the choice for you. Playwright is also the right choice for you if you have fallen in love with a specific test runner or don’t need one at all. With the framework being reasonably new, we can also expect that the community, documentation and framework in general will continue to improve over time.

Cypress:

➕ Easy to understand documentation
➕ A broader community and easier to find answers about specific issues
➕ Easier to understand for people new to testing
➕ You only have to read up on one framework as Cypress has everything included.

➖ Doesn’t support multi-page and third-party implementations.
➖ More extensive and with more dependencies
➖ Generates several example files and folders
➖ You have to re-run tests to run in another browser.

Playwright:

➕ Broader browser support
➕ Fewer dependencies than Cypress
➕ Supports multi-page and third-party implementations
➕ Lets you choose your test runner.
➕ Doesn’t generate any files.
➕ You can run multiple browsers using the same test.

➖ Documentation not as good
➖ Newer and with a smaller community

SpareBank 1 Utvikling

Vi jobber med digitale løsninger hos SpareBank 1.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store