Reinvigorate PHPUnit Test Cases with Parallel Testing
In this post, I’d like to cover our test process when we need to deploy any code into the production environment. Our app works like a bridge or middleware between providers and customers. Providers have some products and customer consumes them on our app. We need to test each provider separately because each has a particular scenario and case. You can imagine that there are some external services and we request them according to our client's requests. So, we become a client across the external services. The endpoints we have written deal with too many network and DB transactions, so we should not forget that the more test scenarios we have, the more time we will lose.
API Test Case
I suppose you’re aware of PHPUnit and there is an extra package named APITestCase to test your API endpoints. APITestCase is a PHPUnit TestCase that will make your life as a Symfony API developer much easier. It extends basic Symfony WebTestCase with some cool features. I strongly recommend you to use this package and write test methods/cases for your app. These test cases drastically decreased our production error. The production app works flawlessly at the moment because of these test implementations for each deployment. APITestCase creates a basic web server in your app and while it’s deploying and it’s able to send HTTP requests to internal API endpoints. These endpoints are already sent to our external services in our scenario. Our purpose is to test whether these endpoints do they work or not properly?
How did we implement it?
Firstly, we were supposed to install the following packages by the Composer. Keep that in mind, we need to have these packages in the dev or test environment. One more thing, we use the Symfony framework for that app.
composer require --dev symfony/test-pack symfony/http-client justinrainbow/json-schemaAfter that, we create a new directory named “tests” at the root of the project, if we don’t have it. Generally, we should have an abstraction for test files if they have many common points. Therefore I created the following abstract class.
Thus, once we extend this class into the normal test files, we will be able to use these services at once. Because these services are used for each provider test file so it doesn’t make sense to initialize them in provider test files one by one.
Distinctly, we request our /balances endpoint and the first test method checks whether everything is or not. The important part is “static::createClient” and it means we’re going to request our internal endpoint that stays in our app. Firstly, we try to match containing some fields in the response array. Then, we check the balance numeric state. And, there should be failed test cases for each successful test method. Therefore we send the wrong parameter in the HTTP request and as expected, we get an error response array from our app. Also, we try to match failed responses by using built-in functions. They’re clearly useful for us. I’d like to run this test file and show you the results. Our command is like the following in the root directory of the project.
vendor/bin/phpunit --testdox tests/FirstProviderTest.php
Actually, I run a test file that belongs to our real test and it has many test cases. As you see, there were 8 test cases in that file. Everything went well and we were able to finish these cases in 13 seconds. In the beginning, everything was going well, but as the number of providers increased, we started to encounter some problems. Why? Because these test cases consume the time a lot by the network and DB transactions. These processes are taking time a lot. Now, we came to our actual problem. We have 10 providers at least and they have 8–10 test cases independently. What would be the test cases' response time in this situation? Let’s check out that.
Engrossing. It considerably runs slower than expected. Think that these providers' number is increasing over time. Memory usage doubled. The time used has increased 10 times. This elapsed time is not acceptable and we started thinking to get rid of these synchronous processes. They don’t need to work like this. So, we found parallel testing by PHPUnit. We are able to run these test files parallel according to the CPU core.
Parallel Testing
They support parallel testing in PHPUnit. We can run synchronous test scenarios asynchronously with this method. I couldn’t find a reason not to use it in our scenario. We use processor cores for this. Provided you have well-written PHPUnit tests, you can drop these packages in your project and start using them with no additional bootstrap or configurations! We have used the following package for this. It is really good that we also have a configuration for the package. Only the command we run in the root directory changes, that’s all because this package actually runs through PHPUnit, so it doesn’t need any extra configuration. Test cases might be a slow process, in particular, once we start talking about consuming a database or automating an API endpoint. To ensure those steps more quickly and quality, we need to be able to run our tests concurrently (at the same time), instead of serially (one after the other).
Why use
paratestover the alternative parallel test runners out there?- Code Coverage report combining. Run your tests in N parallel processes and all the code coverage output will be combined into one report.
- Zero configuration. After composer install, run with
vendor/bin/paratest. That's it!- Flexible. Isolate test files in separate processes or take advantage of WrapperRunner for even faster runs.
To install with composer run the following command. After installation, the binary can be found at vendor/bin/paratest. Run it with --help the option to see a complete list of the available options.
composer require --dev brianium/paratestIt is a package that works on top of PHPUnit and lets us proceed with test cases in parallel without the use of a library. This is an ideal package for functional tests and other long-running processes. To allow full usage of your CPU cores, you should have at least one process per core. More processes allow better resource usage but keep in mind that each process has its own costs for spawning. The default is auto, which means the number of logical CPU cores is set as a number of processes. You might try something like logical CPU cores * 2 (e.g. if you have 8 logical cores, you might try 16), but keep in mind that each process generates a little bit of overhead as well. The default Runner for PHPUnit spawns a new process for each test case (or method in functional mode). This provides the highest compatibility but comes with the cost of many spawned processes and a bootstrapping for each process. Especially when you have a slow bootstrapping in your tests (like a database setup) you should try the WrapperRunner with --runner WrapperRunner. Let’s run the same test file by para test.
# Run all unit tests in 8 parallel processes
vendor/bin/paratest -p8 --runner WrapperRunner
Radiant! For this time, the same 59 test cases are worked in 25 seconds and by 12 MB of memory. It decreased everything smoothly. We have used 8 cores at the same time and it means 8 provider test files are run at the same instead of one by one. It really saves our deployment process. We are very happy to use this.
To Sum Up
Initially, I’d suggest you use the APITestCase package as long as you are an API developer because you cannot test each endpoint for each deployment. APITestCase automates everything for you and gives you peace of mind. But when you use this, you run into latency issues this time as your test files increase. At this point, I recommend parallel testing because it eliminates unnecessary waiting points and uses more memory. Even in our scenario, it accelerated the test processes by at least 250% and we don’t have that many providers yet. I hope it was a useful article for you and I can’t wait to hear your thoughts.
