Metrics are the stuntmen of testing
I just love the old Jackie Chan movies. Can you guess why?
I just recently watched the Civil War movie of the Avengers universe. It was action packed awesomeness all the way till the end.
The list of stunt doubles in the credits was formitable of course.
The thing in Jackie’s movies however is that there are no stunt doubles for the master himself. We always got to see the real thing in action. It still feels priceless.
Post production of modern movies of course makes the ‘priceless’ easily achievable. But even now knowing when we have a stand-in on screen instead of the real thing feels awkward.
Maybe that is why I have so much trouble trusting testing metrics we try to force on our projects.
Testing metrics are always a stand-in for something else. They try to substitute, simplify and make things comparable at the expense of the full experience of the reality.
Metrics that we use are the stuntmen of testing.