Simplified API Test Automation — Introducing Block Framework

Real-world testing challenges go beyond technical complexities. We must innovate and strategize to deliver optimal results. It’s not just about adopting new tools, but fostering a mindset that welcomes creative solutions.

API test automation might seem like a walk in the park for those with experience. After all, there are plenty of tools available. Creating custom code for API checks is often straightforward. This isn’t always the case, though. Sometimes, the established methods just don’t fit with the unique demands of a project, or the bigger picture in the business world.

At Bluecore, we’ve noticed that the usual approaches can be quite limiting, particularly when scaling up or integrating smoothly into existing systems in preview of dynamic requirements. We think outside the box by taking inspiration from LEGO® blocks — known for their simple yet flexible nature, we aim to bring these qualities into our automation framework. This article explores the tangible hurdles one may encounter during API test automation, and puts forth a solution to mitigate them effectively.

Available Strategies to Approach API Testing Automation Projects

How do test automation leaders typically respond when faced with a new API testing project? Generally, they consider two main strategies:

• Utilizing Industry Standard Tools: This involves making use of well-recognized and adaptable tools, like Postman. Such tools are reliable, well-supported, and well-documented, making development and troubleshooting smoother. Plus, they’re easy to learn and use, which speeds up onboarding and script writing. This strategy is usually adopted if the business approves and there are no security or regulatory concerns with the tool.
• Developing an In-House Framework: This strategy involves leveraging open-source technologies and frameworks to create a customized solution. It is particularly favored when there is a need for customization and scalability, especially if there are budget constraints or a preference for in-house development.

As we delve into the above strategies at Bluecore, we discuss the actual requirements, which might echo some challenges you may encounter. Below is a sample conversation:

Program Manager: We’re upgrading to the latest Python version and need to test over 100+ APIs. We need to cover positive and negative test cases for each API. We’re thinking about a phased approach during development sprints. Your thoughts?

QA Lead: A phased approach is wise. For testing, are we considering standard tools or something more tailored?

Program Manager: That’s the question. Standard tools are straightforward, but can they handle this scale?

QA Lead: Postman could manage the basics, but for this scale, a custom framework might offer the flexibility we need. It could be more scalable.

Program Manager: Interesting. And how about managing multiple test environments? They need to be auto-purged regularly.

QA Lead: A custom framework could integrate environment management seamlessly. We’d have control over data and environment specifics.

Program Manager: And there’s the challenge of testing end-to-end workflows, which includes handling API logs in BigQuery and/or in-memory objects.

QA Lead: That’s a complex requirement. Licensed tools might fall short there. Our custom solution could be designed to handle such intricacies.

Program Manager: Okay. Let’s also think about the team. We have two automation testers and two functional testers. One is keen on API testing.

QA Lead: We have a solid foundation. Let’s build a team strategy that leverages their strengths and fosters skill development.

Standard approaches to API automation, despite being tried-and-tested, sometimes fall short in offering the desired level of flexibility. This leads to obstacles in achieving seamless integration and scalability. Using our example project from above, the usual methods (such as making separate libraries for each API client, or setting up integration tests with pytest) may be slow and difficult, especially with the given challenges:

• Scalability: test management of over 100 APIs during a backend upgrade to the latest Python version.
• Customization: the need for a tailored approach, accommodating phased automation and test planning during development sprints.
• Integration: seamless integration into existing systems across multiple, dynamically changing test environments.
• Data Management: efficient management if data in a manner that is both environment- and data-agnostic, while also handling in-memory objects and database interactions.
• Third-Party Integrations: handling interactions with large-scale databases like BigQuery, and in-memory objects like Redis or Spanner.
• Resource Limitations: a compact team with specialized skills, not all equipped with automation testing expertise.
• Skill Development: fostering skill development for team members eager to expand their knowledge in API testing.

Reinventing Strategies with the LEGO® Block Abstract Concept

The modularity of LEGO® blocks allows for adaptable and flexible frameworks.

Faced with these challenges, we sought a more adaptable solution, finding inspiration in LEGO® blocks. These toys, known for their simplicity and flexibility, mirrors what we need in our projects — a method that is both straightforward and adaptable.

Like creating a structure with LEGO®, the Block Framework allows us to build our project with individual, standardized units. These units can easily fit together, making the whole process more straightforward without sacrificing any of the details.

What are Blocks in the Block Framework?

Okay, here’s the scoop — in the Block Framework, a “block” is like a little testing superhero with its own superpower of functionality. These blocks are capable of executing specific actions, such as initiating API requests, validating responses, and even interacting with other blocks seamlessly.

Think of backend services like RestRequest and GRPCRequest as simple, interchangeable blocks, each with standard input and output features. This setup makes orchestrating complex workflows easy, allowing the blocks to work together seamlessly, and manage the intricacies of API testing effectively.

Block Configuration: Everything You Need to Write Test Cases

The Block Configuration serves as your fundamental toolkit in the testing process, where you are in charge of shaping the rules and structure. It facilitates the organization of individual blocks, each designated with a unique role, enabling you to tailor your tests according to your precise preferences.

Here is an example to illustrate the functionality of a standard RESTRequest block:

The Block Configuration demonstrates the metadata, input values, the validations necessary for the test, and the output from the test completion

• Metadata: this section assists in defining the goal of each block. With fields like block name, description, and other attributes, it enables the customization of block execution. It also includes tagging blocks facing application issues and linking them to the respective Jira tasks, facilitating a comprehensive and customized report.
• Input: this section houses all the vital configurations necessary for the block to start processing. The details within this section will fluctuate based upon the block type.
• Output Configurations: this facilitates text and response attribute validations to streamline the verification of crucial elements and response attributes, respectively. Additionally, it provides a secure space for storing key data elements, enhancing workflow coherence.

Block by Block: Enhancing Efficiency with Scriptless Testing

After shaping these components into robust classes, the aim was to foster straightforward collaboration between blocks using uncomplicated, light configurations. To implement this, a data-driven strategy was adopted, leveraging the light and readable JSON to orchestrate block configurations, simplifying test case management and maintenance, particularly with git repositories.

Multiple blocks combine together to generate a JSON end-to-end test

Each JSON file configuration functions as a distinct test case, encompassing one or more building blocks that serve as test steps. These blocks are designed to address specific business objectives. As a result, we optimize the testing process while maintaining a level of flexibility and user-friendliness.

This progress has made the technical aspects simpler, allowing for a smoother testing process and increased confidence in delegating API testing to functional testers. This approach helps broaden our automation initiatives, with functional testers leading the way in an adaptable and scalable testing framework.

Addressing Data and Environment Dependencies

To create a testing environment that is unaffected by particular data or settings, we established a central test data management system. This system leverages dynamic data pulled from static files, ensuring uninterrupted operation — even when environmental data is deleted. Whether these central data files are generated manually or automatically, the integrity of the remaining workflow stays preserved. This is a common pain point for many test teams.

As the framework developed, we employed it to automatically generate these data files, enhancing efficiency and cohesiveness. Utilizing the same JSON configuration files, we documented output data directly within these files, which were then accessed by multiple test cases until deletion.

Putting it all together

As illustrated in the diagram below, the framework integrates several vital components, each with specific roles, to create a streamlined testing process:

The Block Framework can be easily integrated and managed by large test teams and across multiple facets of work.

• Test Driver: this is the front gateway for integrating JSON files into the framework. The driver excels at managing either individual or numerous test cases at the same time, promoting streamlined test execution.
• Data Configuration: a centralized data repository harboring both the test cases and the associated test data. Crafted and managed by test engineers, this module dictates the test workflows and permutations, ensuring seamless operation.
• Block Executor: this is the framework backbone that synthesizes data configurations with JSON templates, orchestrating the workflow according to the specifications laid out in each block configuration. It acts as a mediator, ensuring harmonious interaction between blocks, while managing data configurations seamlessly from INI (initialization) files to JSON configurations.
• Block Factory: This outlines the core capabilities in the form of various functional blocks like REST, gRPC, Spanner, etc. It provides a JSON interface for necessary actions within each component, thereby facilitating smooth functionality
• Common Utils: These are reusable libraries that sustain the framework, encompassing shared logic and functionalities like assertions, data handling, and logging and reporting mechanisms. As a result functionality and efficiency of the entire configuration is enhanced.

By integrating these components seamlessly, we lay the groundwork for a simple yet robust testing framework, promising efficiency and adaptability at every stage of the testing process.

Meeting Expectations: A Seamless Delivery

In just a single quarter, our team integrated over 400 test cases using the Block Framework. What’s remarkable isn’t just the volume, but the approach. These tests form part of our progressive automation strategy: dynamic, adaptable, and requiring minimal maintenance.

With an initial high investment of automation works, the Block Framework pays off over time for efficiency of the test team, particularly when we start to add a large amount of test cases.

Usually, handling this many tests would be a tough job. Thanks to the Block Framework, we’ve made this process simpler. We are reducing the time and effort needed for automation gradually, making it easier for our functional team to expand the automation suites as needed.

Navigating Challenges: The Bluecore Expedition

As the sample project phase wraps up, it’s clear that the new method we used was quite successful. The new approach significantly streamlined the team’s workflow. We were able to tackle complex tasks without getting tangled in technical complexities. Moreover, the automation framework could scale independently, reducing the constant need for oversight from automation engineers.

Our functional testers adapted quickly, taking charge of both creating and executing test cases. This not only made processes more efficient, but also provided a rich learning experience for the team. It empowered the team to independently familiarize themselves with the nuances of the automation framework.

The team is now well equipped to take full responsibility for maintaining the framework. Initially developed for this project, the positive outcome has encouraged us to consider using this approach in other areas as well.

Learning Curve and Anticipated Challenges

The journey towards adopting the Block Framework comes with its fair share of hurdles that may impact the ease of transition and adoption. Key anticipated challenges include:

Adoption of JSON: the reliance on JSON for configuration and test case definitions may not be intuitive for everyone, especially for those primarily acquainted with scripting or programming.

Codeless Test Cases: seasoned testers might find the shift to a codeless setup mundane, affecting their enthusiasm or ownership. However, this framework is designed to streamline testing, allowing for enhanced coverage with reduced effort. This frees up time to quickly tackle Block Framework tasks, and move on to additional framework design needs.

Adapting to Block Structure: mastering the block structure, in defining and configuring blocks for varied testing scenarios, is crucial for leveraging the framework effectively. We do find there is a small learning curve involved.

Transitioning Mindset: a shift in mindset is required to transition from traditional scripting to a structured, modular block-based approach. This may be challenging for some team members.

Additionally, if there are minimal APIs to test, using standard tools (like Postman) might be more practical. It’s essential to weigh the project’s unique needs and context when considering adopting this framework.

Bluecore’s adoption of the Block Framework, while challenging, was strategically aligned with our long-term goals and scalability needs. Block Framework makes these challenges relatively minor in the broader context. Overcoming these challenges truly depend on the requirements at hand. We made the necessary trade-offs, keeping in mind the ultimate efficiency and adaptability benefits that this framework offers.

Reflecting on the Approach

In conclusion, real-world challenges in any testing project go beyond technical complexities. They prompt us to innovate and strategize effectively, and deliver optimal results under given circumstances. The journey isn’t merely about adopting new tools, but fostering a mindset that welcomes creative solutions.

As shown, the Block Framework has proven to be a great problem-solver for our API testing needs. It shows us the importance of thinking strategically, encouraging us to keep tweaking our methods, to move swiftly and wisely in the ever-changing world of automation testing.