Setting Up a Robust Performance and Load Testing Framework with k6

Introduction: Embracing the Need for Speed in Software Performance

In today’s digital landscape, where milliseconds can mean the difference between success and failure, ensuring your software performs under pressure is not just an option — it’s a necessity. Users expect seamless experiences, and search engines favor fast-loading applications. Here, performance testing emerges not just as a task on a checklist, but as a cornerstone of user satisfaction and business success.

Performance testing transcends traditional debugging; it’s about proactively enhancing user experience, ensuring scalability, and maintaining reliability under varying loads. Whether it’s a sudden traffic surge on an e-commerce site during a sale or consistent load on a financial application during trading hours, every application has its crucible moments that define its market standing.

In this guide, we’ll navigate the waters of performance testing, focusing on practical approaches and modern tools that align with today’s development practices. Drawing from real-world examples and best practices illustrated in our k6 Test Template GitHub repository, we aim to bridge the gap between theoretical knowledge and practical application, ensuring a comprehensive understanding of performance and load testing in a software development context.

GitHub - mohsenny/k6-test-template

Chapter 1: Why k6 Stands Out in the Performance Testing Arena

Choosing the right tool for performance testing is pivotal. Among the myriad of options available today, k6 shines for its simplicity, efficiency, and developer-centric approach. Here’s why k6 is a compelling choice for your testing arsenal:

Developer-Friendly

k6 is built with developers in mind. It allows you to write performance tests in JavaScript, a language many are already familiar with. This means you can quickly write tests without the steep learning curve associated with specialized testing languages.

Realistic Load Simulation

k6 excels in simulating real-world load scenarios. From simple tests to complex ones involving stages, ramping up and down users, or testing different parts of your application concurrently, k6 offers a flexible approach to mimic user behavior accurately.

Open Source with Enterprise Support

Being open-source, k6 has a thriving community and a wealth of shared knowledge. It also offers enterprise support for teams needing advanced features and dedicated assistance.

CI/CD Integration

In the era of continuous integration and delivery, k6 seamlessly integrates into CI/CD pipelines, making it possible to automate performance testing alongside your regular deployment processes. This integration ensures performance benchmarks are met with every release.

Detailed Reporting

k6 provides detailed reporting of test results, offering insights into request times, throughput, error rates, and more. These insights are crucial for identifying bottlenecks and making data-driven decisions to improve performance.

Scalability and Cloud Support

k6 can be run on your local machine for development or scaled up using its cloud service for large-scale tests, providing the flexibility to test applications of any size.

By harnessing the power of k6, teams can ensure their applications aren’t just functional but are also optimized for the best user experience under various conditions.

K6 Limitations

While k6 offers robust features for performance testing, it’s important to be aware of certain limitations, especially regarding module imports and script compatibility.

Node Module Import Limitation:

k6 doesn’t support importing Node.js modules directly, primarily due to potential performance issues and negatively affecting the memory allocation for its VUs (Virtual Users). Node.js modules often contain features and dependencies not suited for a performance testing environment, which could impact test efficiency and accuracy.

To overcome this, users typically use raw HTTP(S) imports or bundle their code using tools like Webpack. This ensures only necessary code is included, optimized for k6’s environment.

JavaScript-Only Execution:

k6 is designed to run JavaScript files, not TypeScript or other languages. This constraint is aligned with k6’s goal of streamlined performance testing, as JavaScript is universally compatible and generally more efficient for the engine k6 uses.

Despite these limitations, k6 remains a powerful tool for performance testing. Understanding and adapting to these constraints allows users to leverage k6’s full potential while maintaining efficient and reliable test suites.

Chapter 2: Crafting Your Testing Environment with k6

Setting up an effective testing environment is crucial for accurate and reliable performance testing. In this chapter, we will guide you through the process of configuring your environment using k6, drawing insights from the k6 Test Template repository. This GitHub repository serves as a practical example to complement our discussion, offering you real-world applications of the concepts we cover.

Step 1: Installing k6

Performance testing starts with the right tools. k6, known for its simplicity and power, is our tool of choice. Here’s how to get it up and running:

For Windows Users:

In the k6 Test Template repository, you’ll find a PowerShell script local.ps1 that effortlessly sets up your environment:

# Navigate to the repository's root directory and run:
. .\scripts\setup\local.ps1

This script autoates the installation of k6 and sets up the necessary environment variables, streamlining your setup process.

For Unix-based OS Users:

Similarly, for Unix-based systems, the repository provides a local.sh script:

chmod +x ./scripts/setup/local.sh && source ./scripts/setup/local.sh

Running this script installs k6, configures your environment, runs npm install as well as tsc to compile your typescipt files, ensuring you’re ready to start testing in no time:

# local.sh for Unix
#!/bin/bash

brew install k6
export MY_HOSTNAME="https://jsonplaceholder.typicode.com"
export RUN_TYPE="performance"
npm install
tsc

# local.ps1 for PowerShell
winget install k6 --source winget
$env:MY_HOSTNAME="https://jsonplaceholder.typicode.com";
$env:RUN_TYPE="performance";
npm install
tsc

Step 2: Understanding the Project Structure

A well-organized project structure is key to maintaining efficiency and scalability in testing. The k6 Test Template repository offers a structure that you can adapt or use as is:

│   .gitignore
│   .prettierrc
│   package-lock.json
│   package.json
│   README.md
│   tsconfig.json
│
├───helpers
│   ├───cleaners
│   │   │   common.js
│   │   └───scripts
│   │
│   ├───config
│   │       configure.js
│   │       endpoints.js
│   │       index.js
│   │
│   ├───requests
│   │       jsonPlaceholder.js
│   │
│   └───utils
│           common.ts
│           http.ts
│           notifications.ts
│
├───scenarios
│   │   load.js
│   │
│   ├───example.spec
│   │       jsonPlaceholder.ts
│   │       postmanEcho.ts
│   │
│   └───types
│           custom-types.d.ts
│           setupData.ts
│
└───scripts
    └───setup
            ci.sh
            local.ps1
            local.sh

Helpers: Contains utility scripts and configurations.

• cleaners: Scripts for data cleaning or preparation.
• config: Configuration files like configure.js for setting up environments and endpoints.js for API endpoints.
• requests: Houses scripts like jsonPlaceholder.js for handling specific API requests.
• utils: General utility scripts, including common functions (common.ts), HTTP utilities (http.ts), and anything else.

Scenarios: Test scenarios and types.

• load.js: Defines k6load testing stages.
• example.spec: Specific test cases (e.g., jsonPlaceholder.ts, postmanEcho.ts).
• types: TypeScript declaration files (e.g., custom-types.d.ts) and setup data types (setupData.ts).

Scripts/Setup: Scripts for setting up the testing environment.

• ci.sh, local.ps1, local.sh: Scripts for continuous integration and local setup.

This structure is designed to keep your tests organized and easily manageable. In the mentioned repository, you’ll see examples of setting and using environment variables in scripts. These variables help tailor the testing process to your specific needs without hardcoding values.

Chapter 3: Crafting Requests and Configuring Load in k6 Tests

Performance testing with k6 is not just about running scripts; it’s about accurately simulating user interactions and load patterns. In this chapter, we delve into creating reusable request functions and configuring load scenarios, drawing practical examples from the k6 Test Template repository.

Creating Reusable Request Functions

One of the key principles of efficient testing is reusability, especially when it comes to making HTTP requests. In the helpers/requests/ directory of our template, you'll find examples of modularized request functions. These functions represent different API calls that can be reused across multiple test scripts.

Here’s how you might structure a request function for getting posts from an API:

// helpers/requests/jsonPlaceholder.js
export const getPosts = () => {
  const response = http.get(
    endpoints.jsonPlaceholderPosts.getPosts,
    {
      headers: defaultHeader,
    }
  );
  if (response.status !== 200) {
    // log errors...
  }
};

This function can be imported and used in any test script, ensuring that your API calls are centralized and maintainable.

Configuring Load with load.js

The heart of performance testing lies in accurately simulating real-world load on your application. This is where load.js comes into play. Located in the scenarios/ directory, this file allows you to define various load patterns that your tests can use.

Concept of load.js

load.js acts as a central repository for your load configurations. It defines how many virtual users (VUs) will interact with your application, for how long, and in what pattern. This setup enables you to simulate different environments and usage patterns, from light usage to peak traffic scenarios.

Let’s look at an example load configuration:

// scenarios/load.js

export const loads = {
    'lightLoad': [
        { duration: '5m', target: 10 }, // Simulate 10 users for 5 minutes
    ],
    'heavyLoad': [
        { duration: '10m', target: 100 }, // Ramp up to 100 users over 10 minutes
    ],
};
export function getLoadProfile(profileName) {
    return loads[profileName] || loads['lightLoad'];
}

In your test scripts, you can then import getLoadProfile and use it to set the appropriate load for each scenario:

import { getLoadProfile } from './load.js';

export const options = {
    stages: getLoadProfile('heavyLoad'),
};

Final Chapter: Bringing It All Together with a Practical Test Example

With reusable request functions and configurable load patterns, your k6 tests become both versatile and realistic. You can quickly adapt your tests to different scenarios, from routine checks to stress testing under heavy load.

After laying the groundwork with request functions and load configurations, it’s time to see how everything comes together in a practical test example. This final chapter demonstrates how to integrate the reusable requests and load patterns into a complete test script, drawing from the k6 Test Template repository.

A Comprehensive Test Script

Let’s look at a test script that combines everything we’ve discussed. This script simulates user behavior by making API calls to fetch and create posts, following the load pattern defined in load.js.

Example Script:

import { sleep, group } from 'k6';
import { getLoadProfile } from '../scenarios/load.js';
import * as jsonPlaceholderRequests from '../helpers/requests/jsonPlaceholder.js';

// Set the load profile from load.js
export const options = {
    stages: getLoadProfile('heavyLoad'),
};
export default function () {
    group('Get all Posts', () => {
        jsonPlaceholderRequests.getPosts();
        sleep(1);
    });

    // More scenarios for the same endpoint could follow, i.e.:
    // 
    // group('Get a single Post', () => {
    //   ...
    // });
    // 
    // group('Create a Post', () => {
    //   ...
    // });
    // ...
}

In this script:

• We import our load profile and request functions.
• The group function is used to organize different parts of the user interaction.
• The jsonPlaceholderRequests.getPosts() function is called to simulate fetching posts.
• Sleep intervals (sleep(1)) are added to mimic real user think time between actions.

Analyzing Test Output

After running this test, you’ll get an output detailing various performance metrics. Analyzing these results helps you understand how your application behaves under the simulated load, providing insights into areas that may need optimization.

Output example of executing a k6 test

- data_received: The total amount of data received from the target server during the test. It’s shown in kilobytes and the rate per second.

- data_sent: The total amount of data sent to the target server. This includes all HTTP request data sent by k6.

- group_duration: The average, minimum, median, maximum, 90th percentile, and 95th percentile durations of the named groups in your test script. Groups are a way to organize scenarios in k6.

- http_req_blocked: The time spent blocked before initiating the request. This can include time spent waiting for a free TCP connection from a connection pool if you’re hitting connection limits.

- http_req_connecting: The time spent establishing TCP connections to the server. If this is high, it could indicate network issues or server overload.

- http_req_duration: The total time for the request. This includes sending time, waiting time, and receiving time. The detailed breakdown is provided for expected responses (expected_response).

- http_req_failed: The percentage of failed requests. Ideally, this should be 0%.

- http_req_receiving: The time spent receiving the response from the server after the initial request was sent.

- http_req_sending: The time spent sending the HTTP request to the server. This typically is a small number.

- http_req_tls_handshaking: Time spent performing the TLS handshake. If your request uses HTTPS, this includes the time taken to negotiate the SSL/TLS session.

- http_req_waiting: The time spent waiting for a response from the server (also known as Time to First Byte, TTFB). This doesn’t include the time taken to download the response body.

- http_reqs: The total number of HTTP requests made during the entire test.

- iteration_duration: The time it takes to complete one full iteration of the main function in your script.

- iterations: The total number of times the main function was executed.

- vus: The number of Virtual Users (VUs) actively executing during the current test step.

- vus_max: The maximum number of concurrently active VUs during the test.

Conclusion

Performance testing with k6 is a journey of continuous improvement. As your application grows and evolves, so should your tests. By utilizing a structured approach with reusable components, as demonstrated in the k6 Test Template, you can ensure your application is not only functional but also performs optimally under all conditions. It’s a commitment to quality that pays dividends in user satisfaction and application reliability.