Improve Game Efficiency
The real-time camera freezes for about 16 seconds, while Openpose drained system resources for its calculation. The efficiency is clearly too slow for a game. The first idea we came up with was using multithread.
When button is pressed, function button_coutdown is called and when it finishes, it would call the threading function to create a new thread, which should run in the background without interfering real-time webcam. However, the program was not running in the way we expect it to be, real-time webcam stops working until computer finishes calculating. We soon found that, when Openpose is doing the calculation, it takes up all the remaining CPU resources that the CPU could not share any resource to start a new thread.
When we looked back at the stages the program goes through, we found that by preprocessing all the example/model images and save them locally will reduce a certain amount of time to do calculation. We then looked at the website Openpose provided, it mentioned that the result can be saved as JSON files, but it had not documented the way to achieve that using python code. We discover that numpy arrays can be stored as files using savez() funciton and the key results from calculation are stored in arrays. The savez() function takes two or more parameters, the first one is the location where the file is saved at and the rest will be the arrays that need to be stored.
Using preprocessed images for the game reduces the time where webcam stops working by 9 seconds.
As we were looking for strategies to shorter the duration of the calculation and look deeper into the resources provided the developers of Openpose, we found a document showing the efficiency of Openpose running on Intel CPU versions, and that for GPUs [1]. The projected increase is about 11 times on our machine. It is also likely that CPU resources will not be fully taken, if we use GPU for calculation. The described findings above will likely allow developers to resolve the latency problem.
It is also possible to improve efficiency by switching from the BODY-25 model to the COCO model. The cost, however, is that the result will be less accurate, and the boost in speed is limited.
