Functional verification test framework with Go
After going through the positions of Backend and DevOps, I further explored the testing part of software engineering.
When the engineer found that his brain could not ensure that the program he wrote worked correctly, he decided to solve this problem by writing tests.
Through the advantages of concurrency and scheduler in Go, it can have better performance when testing a large number of server endpoints or complicated communication. In order to solve the various problems encountered in the past and improve the workflow, I considered several issues worth paying attention to when designing this framework.
Statistical Report
The go test function provided by go adopts a very delicate design. Through the *testing.T instance in the parameters, it continuously collects the information in each test case and reports it in the end.
However, it is a unit testing tool, and the only thing that can be returned is the pass or error message. This is enough for a single module as the test object, but it is far from enough for the functional verification test of multiple component integration.
Most of the time, we need the response or behavior given by the software service at each staged checkpoint, rather than whether it passed or a long list of error messages. I still imitate go test and pass in a single instance to continue to collect information through parameters, but use JSON format to provide a more flexible message storage structure.
Group test
The functional verification test for a software service may have many different aspects, such as login, shopping cart, sending messages, etc., not every time you need to carry out a carpet test, and not every department or team must carry out all kinds of tests.
A solution that can centrally manage the code of all test cases, was provided but can also group tests when requested to perform tests. By classifying each test case into a different group and attaching a specific group label to the request body when requesting the test, the function of accurate group testing can be achieved.
Execution performance
Compared with most unit tests executed on the local side, the functional verification test directly tests the running service, so there are definitely many network requests and responses or cases that take a long time.
Rely on the native concurrent design of the go language and the Network Poller mechanism in its scheduler to allow him to perform better than parallelism in the case of large or complex network requests.
Most of the time when the functional verification test needs to be executed is at the end of the CI-CD Pipeline. Since it needs to be triggered frequently, it can be accelerated by tens of seconds, the improved in iteration efficiency is very obvious.
However, this type of acceleration method needs to be established on the premise that each case is independent, and I also hope to use this to constrain the user’s design style.
Security issues
There are many services that require authentication and authorization before they can be used. This is also the case during testing, so it is inevitable to provide some sensitive information such as key, password, token, etc. to the test program.
In order to prevent the test program from being maliciously triggered and executed by unknown external users, in addition to being placed in a private cloud and controlled by whitelisted IP, I have also added some additional user measures.
The first is the Job queue task queue. If the service is still busy, it will first store the received test request in the queue. If the queue is full, it will automatically reject the new request to avoid a large number of malicious tests occupy too many resources in a short time.
The second is the JWT authentication mechanism. Any test request must carry a valid token in the request body. If the validity period expires, you must request the token with the account password again.
Ease of use
Many test tools are used as an entire installation software or depend on the language used for development itself. This leads to the need to install or build an environment that can use the language before the test program can run correctly.
I hope to change the dilemma of this high position limit, so I provide a server framework for developers to write test cases and hand them over to the server. This allows other users to use simple two or three lines of Unix commands and attach the required settings to the request body to drive the test correctly and obtain statistical results.
At the same time, because all the tests are performed uniformly by the server after receiving the request, there is no need to worry about the different execution environments of different testers, and the construction cost of the environment is completely solved.
Epilogue
If you are interested in the framework or wanna give it a try, you can refer to my GitHub repository, and I’ll be very happy if you send a PR instead of just using it. I will also share the design and development experience at COSCUP in Taiwan this year. I am very grateful to them for accepting my submission :)
