Monty Python and the Search for Construct Validity
Throughout history there has been one question which has troubled man time and time again: “Sir! We have found a witch! May we burn her?”
Ah! There are ways of telling if she is a witch. “Why does a witch burn?” She is made of wood, of course. “Does wood sink or float?” Wood floats. “What also floats?” A duck floats! “So, logically, a witch should weigh more than a duck. Take her to the scales!”
This is the village scene presented to us in Monty Python and the Holy Grail. Comedy has a delightful way of showing us reality in satire.
Let us deconstruct the test that has determined the woman is a witch.
If, indeed, there is such a thing as a witch, then cutting her open would show that she is not made of wood. If she be made of wood, then weighing her against a duck would show nothing more than that she weighs more than a duck.
The test of weight would not tell us whether or not the witch would float, because it does not tell us how dense the witch is. Density would not, alone, tell us whether or not the witch was made of wood and does not explain why a witch would burn.
The test is therefore not valid for determining whether or not the woman is a witch. However, it would make a good test to understand if the villagers are intellectually lacking. (They are.)
This is also a good test to examine the concept of face validity. Validity in psychological testing is of the utmost importance as tests are sometimes used as projectives for behaviour.
In this case the test was used as a projective to determine whether or not the woman was a witch. There was very little other proof that the woman was a witch, although she was dressed as a witch (by the villagers) and was accused of turning a man into a newt (though he “got better”).
Face validity is defined as the extent to which a test is believed to test what it is meant to test. In this case, the villagers and Sir Knight, alike, believed that the test would tell them if she was a witch.
The problem was that the test did not have construct validity. Construct validity is the extent to which a test measures what it claims to measure.
Many tests of justice throughout history have lacked construct validity, if there were tests at all. Prejudice, inequality and power-hunger often determined that women were witches, causing them to be put through medieval trials of justice (ordeals of fire, water, ordeal of the cross, ingestion, poison, boiling oil, etc…). The issue is that without construct validity, psychological tests are no better than practices of justice of the middle-ages.
We should be careful in perusing the truth, for some may be burned and others turned into newts.