Building a Robust Automation Framework in Playwright (Typescript version)

NOTE: This article is for people who already know what is a playwright and want to improve their automation framework.

As a Software Development Engineer in Test (SDET), the choices we make when building an automation framework go beyond just technical preferences — they significantly influence the effectiveness and quality of our testing. Every decision, from tool selection to testing practices, holds the power to make a lasting impact on our testing efforts. Understanding the rationale behind each decision helps create long-term solutions.

In the world of automation frameworks, various complexities arise due to the diverse landscapes of projects. Factors such as code-hosting and managing approaches, ranging from the dichotomy of multi vs. mono repo setups to the intricate architecture of web applications — spanning Single Page Applications (SPA), Micro Frontends, microservices, Event-Driven architectures, and beyond — all contribute to the nuanced tapestry of challenges we face.

Despite the intricacies of automation framework setups, this article takes a focused approach. We explore best practices for building/designing JavaScript/Typescript-based automated testing frameworks, specifically concentrating on UI, API, and End-to-End tests hosted in a single repository. The spotlight is on practices that seamlessly retrieve test data from databases, mitigating the need for manual interventions across diverse environments. In the following sections, we will navigate through the key principles and strategies that empower the creation of a robust and adaptable automation framework in Playwright with a TypeScript flavor.

The article contains:
Eslint/Prettier setup, Managing environment variables and secrets, Framework structure, Writing Code/Tests — Best Practices, Handling test data, Grouping and Running tests

0. Install Playwright

Installed playwright project in typescript, and you see the following file structure:

playwright.config.ts
package.json
package-lock.json
tests/
  example.spec.ts
tests-examples/
  demo-todo-app.spec.ts

1. Setting up ESLint and Prettier for Code Consistency

In the dynamic landscape of JavaScript and Typescript, where flexibility often translates into potential pitfalls, establishing a robust foundation becomes paramount. Here’s why setting up ESLint and Prettier should be your initial step:

Why ESLint?

JavaScript is a dynamically typed language, and sometimes that can lead to less-than-optimal code. That’s where ESLint comes in handy — it’s a powerful linting tool designed to analyze your code, pinpoint issues, and guide you in rectifying potential problems in both JavaScript (JS) and Typescript (TS) code. By integrating ESLint into your project, you not only ensure adherence to coding standards but also elevate the overall code quality.

Why Prettier?

While ESLint deals with code quality, Prettier focuses on maintaining a consistent style throughout your codebase. In collaborative environments, maintaining a unified style guide becomes pivotal to avoid discrepancies, especially when reviewing pull requests (PRs). Prettier steps in as the enforcer, diligently formatting your code and eliminating concerns over trivial matters like white space or semi-colon nuances. Consider adopting established style guides such as Airbnb to elevate your coding practices to the industry’s best standards.

When you weave ESLint and Prettier into your workflow, you’re building a strong defense against common issues, ensuring code consistency, and fostering collaborative coding experiences that go beyond individual styles.

Let’s dive into the seamless integration of these tools into your Playwright automation framework.

Implementation:
1. Install dependencies:

# ESLint and TypeScript dependencies
npm install --save-dev eslint typescript @typescript-eslint/parser @typescript-eslint/eslint-plugin

# Prettier and ESLint Prettier Plugin
npm install --save-dev prettier eslint-config-prettier eslint-plugin-prettier

# Airbnb style guide
npx install-peerdeps --dev eslint-config-airbnb-base

2. Create a Typescript configuration file(e.g., .tsconfig.json):

{
  "compilerOptions": {
    "target": "es2016",
    "module": "commonjs",
    "baseUrl": "./",
    "paths": {
      "*": ["./src/*"]
    },
    "outDir": "./dist",
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "skipLibCheck": true
  },
  "include": ["src", "./.eslintrc.js", "./playwright.config.ts"],
  "exclude": ["node_modules", "dist"]
}

3. Create an ESLint configuration file (e.g., .eslintrc.js):

module.exports = {
  root: true,
  plugins: ['@typescript-eslint', 'import', 'prettier'],
  extends: [
    'airbnb-typescript/base',
    'prettier',
    'plugin:@typescript-eslint/recommended',
    'plugin:import/typescript',
  ],
  parser: '@typescript-eslint/parser',
  parserOptions: {
    project: './tsconfig.json',
  },
};

4. Create a Prettier configuration file (e.g., .prettierrc):

{
  "printWidth": 80,
  "tabWidth": 2,
  "useTabs": false,
  "semi": true,
  "singleQuote": true,
  "trailingComma": "es5",
  "bracketSpacing": true,
  "jsxSingleQuote": false,
  "arrowParens": "always",
  "proseWrap": "never",
  "htmlWhitespaceSensitivity": "strict",
  "endOfLine": "lf"
}

2. Managing Environment Variables and Secrets

In the intricate world of testing across multiple environments, effectively managing environment variables becomes a cornerstone of a well-organized automation framework. Let’s explore the why and how of this critical step:

Leveraging Environment Variables

In the realm of automation, setting variables at the operating system level — known as environment variables — provides a powerful mechanism for adapting to diverse testing landscapes. Given that the Application Under Test (AUT) traverses various environments, from development (DEV) and quality assurance (QA) to Production (PROD), it’s essential to encapsulate environment-specific values, such as the baseUrl, in a centralized location. The go-to solution? The .env file.

The Role of Dotenv

Enter Dotenv, a Node.js library, and other alternatives that simplify the loading and management of environment variables. By storing configuration details in the .env file, you create a centralized repository for environment-specific values, streamlining configuration changes across different testing stages.

Safeguarding Secrets

Now, the elephant in the room — secrets. While it may be tempting to hardcode them as environment variables, the safest route lies in a more sophisticated approach. Consider the cloud, where platforms like AWS or Azure offer Key Vault services explicitly designed for securing and managing secrets.

Why Key Vault?

Modern applications housed in cloud environments recommend leveraging services like Key Vault. These services provide a secure vault for storing sensitive information, advocating practices like key rotation every two years to meet cryptographic standards. Automating key rotation further ensures robust security, with some companies opting for monthly key changes.

The Cautionary Tale of Hardcoding Secrets

Hardcoding secrets, while a quick fix, is neither secure nor sustainable. It not only exposes sensitive information but also poses a challenge in keeping up with dynamic changes. To sidestep these pitfalls, the recommendation is clear: avoid storing secrets locally and instead fetch them dynamically from the Key Vault.

The Golden Rule: No Secrets in Repositories

Never upload secrets to your repository. If secrets are part of your setup, give them a VIP pass to your .gitignore file. This way, they stay secluded from version control, keeping your sensitive information locked down.

Note: We’ll read secrets from the .env file in this example because setting up the key vault approach can be a separate article by itself.

Implementation:

1. Create an ./environments folder in the root
2. Create a [environment name].env file for each deployment stage
3. Set a new NODE_ENV environment variable on your local machine
4. Update playwright.config.ts to load environment variables

// playwright.config.ts

import * as dotenv from 'dotenv';

switch(process.env.NODE_ENV){
  case 'local': dotenv.config({path: './environments/local.env'}); break;
  case 'dev': dotenv.config({path: './environments/dev.env'}); break;
  case 'qa': dotenv.config({path: './environments/qa.env'}); break;
  default: dotenv.config({path: './environments/qa.env'});
}

5. Add scripts in the package.json to run tests in needed environments

// package.json

"scripts": {
  // ...
  "test": "echo %NODE_ENV% && npx playwright test",
  // ...
}

What problem do we have with this approach?
Our current script has a hiccup — It demands a manual update of the NODE_ENV variable every time we switch testing environments. Not exactly a seamless experience, right? What if we could streamline this process by creating environment-specific commands?
Crafting a Fix
Picture this: we create special commands that not only set NODE_ENV automatically but also seamlessly run tests for the chosen environment. To achieve this goal, we’ll install cross-env package (or its alternatives), then update the scripts as shown below:

// package.json

"scripts": {
  // ...
  "test": "echo %NODE_ENV% && npx playwright test",
  "test:local": "cross-env NODE_ENV='local' npm run test",
  "test:dev": "cross-env NODE_ENV='dev' npm run test",
  "test:qa": "cross-env NODE_ENV='qa' npm run test",
  // ...
}

With these commands, we wave goodbye to manual hassles. Now, switching testing environments is a breeze — just fire up the corresponding command. Say hello to a smoother Playwright automation framework!

3. Restructure the Framework

In our Playwright automation journey, a smart folder revamp is on the horizon. As we navigate UI, API, and End-to-End testing, a well-thought-out structure isn’t just a cosmetic upgrade — it’s a practical move for clarity and simplicity. By assigning dedicated spaces for each test type, we’re gearing up for efficiency and easy management. In this section, let’s unravel why this restructuring matters and guide you through shaping an organized testing landscape that aligns seamlessly with our diverse testing adventures.

Revamping the Test Folder

Start by bundling all things test-related into a neat folder — call it src (short and sweet). Here, it's not just tests but everything they need, like utilities and data. Modify the playwright.config.ts:

// playwright.config.ts

export default defineConfig({
  // ...
  testDir: './src'
  // ...
)}

Special Homes for Test Types

Now, dig deeper. Create dedicated subfolders within ./src for different test types—api, ui, and e2e. This separation ensures each test tribe has its space, ready for independent management and execution.

Utility Hideout

For universal functions that lend a hand to all tests — picture database setups and query runs — stash them in the ./src/utilities spot. Having a common hub keeps things tidy and simple.

Spotlight on Global Setup and Teardown

Meet the VIPs — the global setup and teardown scripts. These scripts roll out the red carpet once before and after all tests. They deserve their own corner, right in the ./src folder.

In the end, the framework structure will look like the following:

📦 your-automation-framework
 ┣ 📂 environments
 ┃ ┣ 📜 local.env
 ┃ ┣ 📜 dev.env
 ┃ ┗ 📜 qa.env
 ┣ 📂 node_modules
 ┣ 📂 src
 ┃ ┣ 📂 globals
 ┃ ┃ ┣ 📜 global-setup.ts
 ┃ ┃ ┣ 📜 healthcheck.setup.ts
 ┃ ┃ ┗ 📜 global-teardown.ts
 ┃ ┣ 📂 api
 ┃ ┃ ┣ 📂 utils
 ┃ ┃ ┃ ┣ 📜 urlBuilder.ts
 ┃ ┃ ┃ ┣ 📜 payloadBuilder.ts
 ┃ ┃ ┃ ┣ 📜 types.ts
 ┃ ┃ ┃ ┗ 📜 dbQueries.ts
 ┃ ┃ ┗ 📜 apiTest.spec.ts
 ┃ ┣ 📂 ui
 ┃ ┃ ┣ 📂 pages
 ┃ ┃ ┃ ┗ 📜 home.page.ts
 ┃ ┃ ┣ 📂 data
 ┃ ┃ ┃ ┗ 📜 mocks.ts
 ┃ ┃ ┣ 📂 tests
 ┃ ┃ ┃ ┣ 📜 loginUI.spec.ts
 ┃ ┃ ┃ ┗ 📜 homeUI.spec.ts
 ┃ ┃ ┗ 📂 utils
 ┃ ┃   ┣ 📜 uiHelpers.ts
 ┃ ┃   ┣ 📜 uiConstants.ts
 ┃ ┃   ┗ 📜 uiUtils.ts
 ┃ ┣ 📂 e2e
 ┃ ┃ ┣ 📂 pages
 ┃ ┃ ┣ 📂 tests
 ┃ ┃ ┗ 📂 utils
 ┃ ┗ 📂 utils
 ┃   ┣ 📜 dbUtils.ts
 ┃   ┣ 📜 types.ts
 ┃   ┣ 📜 constants.ts
 ┃   ┗ 📜 utils.ts
 ┣ 📜 .eslintrc.js
 ┣ 📜 .prettierrc.js
 ┣ 📜 .gitignore
 ┣ 📜 tsconfig.json
 ┣ 📜 playwright.config.ts
 ┣ 📜 package.json
 ┣ 📜 package-lock.json
 ┗ 📜 README.md

This revamped structure sets the stage for smooth test runs and easy upkeep. Let’s dive into the nitty-gritty of running these tests in the “Grouping and Running Tests” section.

4. Writing Codes/Tests

Now, let’s dive into the exciting realm of writing actual code. To ensure our code is not just functional but exemplary, it’s crucial to understand what qualifies as “bad” or “good” code.

How can we write a “good” code? Where can we start?

1. Leveraging Clean Code Principles: The renowned “Clean Code” by Robert Cecil Martin is an invaluable resource for crafting code that stands the test of time. While we may not all have the luxury of time to devour the entire book, bite-sized articles like “Clean Coding for Beginners” or “Clean Code | Functions” offer digestible insights.
2. Embracing Design Principles: Design principles such as DRY(Don’t Repeat Yourself), KISS(Keep It Simple, Stupid), and SOLID provide a sturdy foundation. Following these principles might seem like a no-brainer, but their impact on code clarity and maintainability is undeniable.
3. Mastering Data Structures: In the JavaScript coding landscape, the go-to is often Arrays, but unlocking the full potential means acquainting ourselves with other gems like Sets, Maps, and more. It’s not just about convenience; using the right data structure ensures optimal performance.

Best Practices in Test Authoring:

1. Consult Playwright’s Best Practices: Familiarize yourself with the best practices outlined by the Playwright for effective test creation.
2. Adopt Page Object Models (POM) for UI Testing: Enhance maintainability by structuring UI tests using Page Object Models.
3. Create isolated test cases: Each test case should be independent.
4. Write Meaningful Test Case Titles: Make your test case titles descriptive and meaningful.
5. Follow the AAA (Arrange-Act-Assert) Pattern: Align your Test-Driven Development (TDD) approach with the clarity of Arrange, Act, and Assert.
6. Maintain Cleanliness: Separate additional logic from tests for a tidy and focused codebase.

Implementation:

As an example, let’s automate one of the test cases for a /search API endpoint where search items can be anything (claim, product, etc.).

// ./src/api/search.spec.ts
import { test } from '@playwright/test';
import { getSearchUrl } from './utils/urlBuilder';
import { getSearchTestInputs, getSearchData } from './utils/dbData';
import { getSearchPayload } from './utils/payloadBuilder'

test.describe('Test Search Endpoint: ', () => {
  const initialInputs: ISearchRequest = {
    //... contains request default and initial values
  }
  // build URL: baseUrl+path+apiVersion+...+endpoint
  const url = getSearchUrl();
  test.beforeAll( async ({}) => {
    // Get test inputs from db
    const testInputs = await getSearchTestInputs();
    initialInputs.clientId = testInputs.id;
  });
 
  test('validate getting items by type', async ({ request }) =>{
    // Arrange
    const requestInputs: ISearchRequest = { ...initialInputs, type: 'T'}
    const payload = getSearchPayload(requestInputs);
    const expectedData: ISearchItem[] = await getSearchData(requestInputs);
    
    // Act
    const response = await request.post(url, payload)
    
    // Assert
    expect(response.status()).toBe(200);
    
    const body: ISearchItem[] = await response.json();
    expect(body.length).toEqual(expectedData.length);
  })
})

Explanation:

• Initial Inputs: The initialInputs object is a clean way to manage request defaults and initial values. By using interfaces to structure this object, we ensure type safety and readability. By spreading in requestInputs, we can keep default values unmodified.
• Url: As we’re testing a singular endpoint to adhere to the principle of avoiding code duplication (DRY), it is configured as a variable at the suite level
• BeforeAll Hook: The beforeAll hook dynamically fetches test data from the database, reducing manual intervention and ensuring a fresh set of data for each test run.
• Utility Functions: urlBuilder, payloadBuilder, and dbData are utilities that enhance readability and isolate additional logic. These utilities contribute to a flexible and maintainable framework by encapsulating code not directly related to the test logic.

My following article contains detailed explanation for each of them:

Building a Robust API Testing Framework in Playwright(Typescript version)

4. Handling Test Data with Playwright Fixtures

The most effective approach to managing test data in an automation framework is done by dynamically retrieving it from a data source. This method ensures that your tests remain adaptable and environment-agnostic. Fetching test data dynamically reduces the need for manual interventions, enhances the freshness of data for each test run, and creates a framework that can seamlessly transition across different testing environments. This not only improves the reliability and accuracy of your tests but also promotes maintainability and scalability in the long run.

I showed the traditional way of handling test data through the dbData utility in the previous code example, but is there a better way to handle it in Playwright? — Yes, by creating playwright fixtures. Playwright fixtures are primarily designed to provide setup and teardown functionality for tests, allowing you to perform actions before and after tests run. While fixtures can be used to fetch data and set up the testing environment, using them to directly retrieve expected data for assertions might not align with their intended purpose.

Remember, the choice of the method depends on the specific needs and scale of your automation framework. Playwright Fixtures offer a good balance between simplicity and functionality for many scenarios.

Let’s take our previous example and refactor it by using fixtures. Start by creating a fixtures file and then use it in the test.

// fixtures.ts
import { fixtures } from '@playwright/test';
import { fetchSearchDataFromDB } from './path-to-your-db-utils';

// Define a fixture to fetch user data from the database
const searchDataFromDBFixture = fixtures.fixture(async ({}, test) => {
  // Assuming you have a client ID as a required field, you should search for data related to that.
  // Query db to get a valid client data (ID) for testing
  const client = await fetchClientDataFromDb();
  
  const initialInputsSeach: ISearchRequest = {
    clientId: client.id
    //... contains request default and initial values
  }
  // Provide the fetched user data to the test
  await test(initialInputsSeach);
});
export { searchDataFromDBFixture };

// search.spec.ts
import { test } from '@playwright/test';
import { searchDataFromDBFixture } from './path-to-your-fixtures';
import { getSearchUrl } from './utils/urlBuilder';
import { getSearchPayload } from './utils/payloadBuilder'
test.describe('Test Search Endpoint: ', () => {
  // Use the fixture to fetch user data from the data source
  test.use(searchDataFromDBFixture);
  
  // build URL: baseUrl+path+apiVersion+...+endpoint
  const url = urlBuilder.getSearchUrl();
  
  test('validate getting items by type', async ({ request, initialInputsSeach }) =>{
    // Arrange
    const requestInputs: ISearchRequest = { ...initialInputsSeach, type: 'T'}
    const payload = getSearchPayload(requestInputs);
    
    // Act
    const response = await request.post(url, payload)
    const body: ISearchItem[] = await response.json();
    
    // Assert
  })
})

In this example:

• userDataFromDBFixture is a fixture that fetches user data from the database.
• The test.use(userDataFromDBFixture) line in your test ensures that the test has access to the fetched user data.

Using fixtures in this way helps you keep your test data retrieval logic separate, making it easier to manage and update.

5. Grouping and Running Tests in Playwright

Running tests highly depends on how we structure our tests. Therefore, we created a separate folder for each type of test in Section#3.

Running All API Tests:

To execute all API tests against the QA environment, a simple modification to our scripts in package.json does the trick:

// package.json

"scripts": {
  // ...
  "test:api:qa": "cross-env NODE_ENV='qa' npx playwright test ./src/api",
  // ...
}

Running Specific API Tests:

What if we want to run specific API tests and not the entire suite? Here are two strategies:
1. Separating Tests by Feature
Create individual folders for each feature and run tests accordingly:

npx playwright test ./src/api/featureName

or

npx playwright test ./src/api/feature.spec.ts

2. Grouping Tests (for Regression or Sanity Suites)

Tag test cases to form suites (e.g., regression or sanity) and execute them selectively:

// example.spec.ts
import { test, expect } from '@playwright/test';

test('Test login page @regression', async ({ page }) => {
  // ...
});

test('Test full report @sanity', async ({ page }) => {
  // ...
});

To run only the (API) regression tests:

npx playwright test ./src/api --grep @regression

Updating Scripts for Targeted Runs:

Refining our scripts in package.json allows us to tailor our test executions:

// package.json

"scripts": {
  // ...
  "test:qa:api:regression": "cross-env NODE_ENV='qa' npx playwright test ./src/api --grep @regression",
  // ...
}

CI-CD Pipeline Integration:

Our tests seamlessly integrate into CI-CD pipelines by providing the required environment variables and secrets, ensuring a smooth transition from development to deployment. With this comprehensive setup, our Playwright automation framework is ready for robust testing in any environment.

Bonus: VSCode Extensions for Playwright

VSCode extensions are incredibly useful because they enhance the functionality of Visual Studio Code, providing additional features, tools, and language support. They cater to the specific needs of developers and help streamline various tasks, making the coding experience more efficient and enjoyable.

Here are some VSCode extensions that make your life easier:

1. JavaScript and Typescript Nightly: It enables the nightly build of TypeScript (typescript@next) as VS Code's built-in TypeScript version is used to power JavaScript and TypeScript IntelliSense.
2. Playwright Test for VSCode (from Microsoft):
   It can Install Playwright, Run tests with a single click, Run Multiple Tests, Show browsers, Pick locators, Debug step-by-step, explore locators, Tune locators, Record new tests, Record at cursor
3. Eslint and Prettier
4. Path Intellisense: It autocompletes filenames.

Conclusion

In summary, crafting an automation framework in JavaScript or Typescript is a nuanced endeavor that demands meticulous attention to detail. The integration of tools like ES-Lint and Prettier serves as a protective shield, ensuring code quality and consistency. Strategic folder structuring, coupled with adept use of environment variables, empowers us to execute tests with flexibility, minimizing manual interventions. Leveraging dynamic test data from data sources not only enhances testing but also renders our code environment-agnostic. Embracing coding best practices and understanding the rationale behind them elevates us, as engineers, to foster a culture of excellence in automation framework development.

Automation Framwork Template codes can be found here.