Chat GPT: A Game-Changing Tool for Quality Assurance Professionals
Last week we saw a big rise in the use of ChatGPT and its capabilities and I wanted to investigate how GPT usage could improve QA processes and streamline some of the work we currently do and what we can expect in the future.
To understand the potential of GPT, let’s first take a look at what it is and the various QA roles it can potentially tackle.
What is GPT?
GPT-3 (short for “Generative Pretrained Transformer 3”) is a large language model developed by OpenAI. It is capable of generating human-like text, and can be used for a wide range of natural language processing tasks, such as translation, summarization, and text generation. GPT-3 is trained on a massive amount of text data, which allows it to generate text that is often difficult to distinguish from text written by a human. It is one of the most advanced language models currently available, and has been used in a variety of applications.
What are some of the QA Engineer roles?
A QA engineer is responsible for ensuring the overall quality of a product or service through the use of automation and manual testing. This typically involves developing and implementing automation testing strategies and plans, as well as executing tests using specialized tools and frameworks.
Some other potential roles and responsibilities of a QA engineer could include:
- Collaborating with other teams, such as development and customer support, to understand requirements and priorities, and to identify opportunities for automation.
- Developing and maintaining automation testing frameworks and tools.
- Conducting regular reviews of requirements, design documents, and other project artifacts to ensure that automation testing is being properly integrated into the overall quality assurance process.
- Providing guidance and mentorship to other team members on best practices for automation testing.
- Participating in the development of quality standards and processes, and ensuring that they are followed consistently in the context of automation testing.
- Monitoring and analyzing metrics, such as test coverage and execution time, to assess the effectiveness and efficiency of the automation testing effort.
How can GPT help streamline some of these tasks?
In this article I will only focus on 2 tasks, tests documentation and automation.
Generating Test Documentation
Writing tests from acceptance criteria is a crucial part of the quality assurance process. As a QA engineer, it is your job to ensure that the product being developed meets the requirements and expectations of the customer. One of the ways you can do this is by creating and executing tests based on the acceptance criteria provided by the product team or other stakeholders. These then get broken down into test steps and are usually written in TestRail or another cloud based service although some may still use excel.
Now the whole process is fairly simple however it is very time consuming as you would also need to maintain them constantly. So let’s have a look at how we could streamline some of this process using GPT.
If we imagine that this is our basic Jira ticket, instead of breaking this down into different steps as we would normally do, we can simply copy this into the ChatGPT and let it generate the test steps for us.
Acceptance criteria
“As a user I should be able to sign up to the application”
Screen fields
name, surname, email, password
Functionality
Verification email is sent and must be accepted
GPT Results
1. Navigate to the sign-up page of the application.
2. Verify that the sign-up form is displayed and includes the fields for name, surname, email, and password.
3. Enter a valid name, surname, email, and password in the corresponding fields.
4. Click the “Sign up” button to submit the form.
5. Verify that the user is redirected to the email verify screen.
6. Check the email address provided during sign-up to verify that a confirmation email was received.
7. Click the confirmation link in the email to complete the sign-up process.
8. Verify that the user can now log in to the application using the email and password provided during sign-up.
9. Attempt to sign up again using the same email address, but with a different password.
10. Verify that an error message is displayed, indicating that the email address is already in use.
11. Repeat the test using different combinations of valid and invalid input for the name, surname, email, and password fields to ensure that the sign-up process is functioning properly
As you can see, GPT has generated all of the base steps for this test case automatically. This saves time and effort for the QA engineer, who would otherwise need to write out each step manually. Additionally, these generated steps can be used at the requirement stage to create more detailed and accurate acceptance criteria. This can help to ensure that the product being developed meets the needs and expectations of the customer.
Another quick example of generating error cases
User: “can you generate all negative scenarios as well”
GPT Results
1. Leave one or more of the required fields (name, surname, email, password) blank, and attempt to sign up.
2. Enter an invalid email address (e.g. a string of characters that does not include the “@” symbol) and attempt to sign up.
3. Enter a password that does not meet the minimum requirements (e.g. it is too short or does not include a mix of letters, numbers, and special characters) and attempt to sign up.
Given the output we can see that GPT is not only capable of generating the steps for a test case, but it can also generate different test scenarios based on the provided requirements. In this case, we used a very generic query to generate negative scenarios, but you can also provide more specific requirements to create even more targeted tests.
For example, if you know that a certain feature is prone to certain types of defects, you can use GPT to generate test cases that specifically target those defects. This can help to ensure that the tests are as comprehensive and effective as possible, and that all potential issues are identified and addressed.
Theres actually a lot more to this but I think this small example displays enough of its potential.
Generating Cypress Automation Test Scripts
Next let’s take a look at how we can use GPT to create automated test scripts more efficiently. We can use the same sign up example from above, the framework in this case will be Cypress.
As we already have the acceptance criteria and the specific fields we can simply ask it to generate cypress tests based on that.
User: “Can you generate a cypress test from the above requirements” (Note this is exactly the wording I used)
GPT Results
describe('Sign-up functionality', () => {
it('Should allow users to sign up', () => {
// Define the URL of the sign-up page
const signupUrl = 'https://my-application.com/signup';
// Navigate to the sign-up page
cy.visit(signupUrl);
// Verify that the sign-up form is displayed and includes the correct fields
cy.get('form#signup')
.should('be.visible')
.within(() => {
cy.get('input#name').should('have.attr', 'type', 'text');
cy.get('input#surname').should('have.attr', 'type', 'text');
cy.get('input#email').should('have.attr', 'type', 'email');
cy.get('input#password').should('have.attr', 'type', 'password');
});
// Enter valid input in the sign-up form and submit it
cy.get('form#signup')
.within(() => {
cy.get('input#name').type('John');
cy.get('input#surname').type('Doe');
cy.get('input#email').type('john.doe@example.com');
cy.get('input#password').type('P@ssword1');
cy.get('button[type=submit]').click();
});We can see here it generated a simple Cypress test to sign up, however there’s more we can do. Let’s say that the hardcoded data can be reused somewhere else, we can ask the GPT to fix that.
GPT Result
const signupData = {
name: 'John',
surname: 'Doe',
email: 'john.doe@example.com',
password: 'P@ssword1'
};
cy.get('input#name').type(signupData.name);
cy.get('input#surname').type(signupData.surname);
cy.get('input#email').type(signupData.email);
cy.get('input#password').type(signupData.password);
cy.get('button[type=submit]').click();In the example shown above, GPT was able to quickly move the hardcoded values into a constant and update the original test code to use those constants instead. Dont get me wrong this is not perfect as it takes sometime to be able to describe what you want to achieve however this type of refactoring can be useful for making the test code more modular and reusable. Additionally, by providing a set of requirements and a sample of the desired test code, it can generate test scripts that are accurate and well-structured. This can save time and effort for the QA engineer, who would otherwise need to write the scripts manually.
Overall, the use of GPT in QA engineering can greatly simplify and streamline the process of creating automated test scripts. It can help to ensure that the scripts are accurate, comprehensive, and easy to understand, and that they contribute to the overall quality of the product being developed.
And just to add, this is only a small part of what GPT can offer to QA engineers. It can also be used for tasks such as creating and maintaining test data, analyzing test results, and providing insights and recommendations for improving the quality of the product being developed, saving countless hours, which I will also cover in the next article.
What does the future look like?
As GPT technology becomes more widely adopted and integrated into various tools and platforms, it is likely to have a significant impact on the work of software engineers. In the future, QA engineers may be able to use GPT-powered tools to quickly and easily generate comprehensive test scripts and test scenarios based on specific requirements and specifications. These tools may also be able to analyze test results and provide insights and recommendations for improving the quality.
In addition, GPT-powered tools may also be able to integrate seamlessly with popular management platforms such as Jira and TestRail, allowing engineers to easily generate and manage tests within the context of their existing workflows.
Overall, the future of GPT in QA engineering looks promising, with the potential to greatly simplify and streamline many of the tasks that are essential to ensuring the quality of a product or service. Try it out and see how GPT can improve your QA process.
Thank you for reading.
Tools Mentioned
ChatGPT https://openai.com/blog/chatgpt/
Cypress https://www.cypress.io/
TestRail https://www.gurock.com/testrail/
