Stress-ng CPU Benchmark Research
Background
Currently, we evaluating the Microsoft VirtualCline (https://github.com/microsoft/VirtualClient) tests and trying to introduce some test cases into our PnP test list.
Stress-ng Introduction
The main purpose of the stress-ng tool is to stress test the various components of the system to evaluate the robustness of the system and identify potential weaknesses.
And stress-ng tool can measure stress test throughput by measuring the bogo operations per second as a micro-benchmark. The test outcomes are not precise, but they provide a rough estimate of the performance.
MS VS uses the following command to execute stress-ng to measure performance.
./stress-ng --cpu $CORES --timeout 60 --metrics --yaml output.yamlThis command means running $CORES (core count, 8380 2 sockets platform will be 160) instances of the CPU stressor for 60 seconds and summary performance metrics output to the “output.yaml” files.
The CPU stressor has over 75 different methods, so plenty of different ways to exercise the CPU sequentially: floating point, integer, vector math, mixed math, etc. Each CPU stressor will run on one thread. So this command will run a CPU stressor on each logical core.
The picture above is the output result of one test, and the table below is the explanation of these indicators.
Through analysis, we can know only “bogo ops/s (usr+sys time)” is suitable as a PnP indicator.
By analyzing the 100 times execution results, we can know even if we set the execution time to 60s, the actual time may be less than 60s.
The “bogo ops/s (usr+sys time)” will be affected by the actual execution time, we can find the “bogo ops/s (usr+sys time)” will decrease when execution time increases as below table.
Therefore, when collecting stress-ng result data, it’s necessary to ensure the actual time is the 60s to ensure the consistency of the performance behavior.
If we can ensure the test result’s actual execution time is close to 60s, then the value of “bogo ops/s (usr+sys time)” is stable. The CV% and Range% are small, so we can use our default CV% and difference < 1% PnP criteria.
