Reliability, yawn…

OK, so reliability is not the most exciting topic. I never promised it would be, did I? Then again, it’s not something we can ignore. And if you think deeply about it I promise it’s a lot more interesting than you might think.

We talked today about measuring Dominance with the Rorschach…actually, I merely used this as a placeholder in the example, any construct and test could have been used. Right away there was a question about Dominance…what is it? It was obvious that the student would have preferred to move to a theoretical discussion rather than talking about the details of Generalizability Theory, which isn’t all that exciting either (I readily admit). But that’s not what we were there for today…

Students get nervous as soon as they see a few formulas (“will there be math on the midterm? do we have to know all the formulas?). The numbers are important….it’s more or less what separates “testing” from journalism. The whole process is about turning ideas (“psychological constructs”) into methods that produce meaningful and potentially useful information. And that for the most part means numbers.

But before we get to the numbers we need to think about the construct. If we were really going to go about measuring Dominance, we would first have to carefully define what (we think) it is, and how and why we think the methods we choose to measure it are relevant. And as we talked about today, we would have to decide in advance about things like stability and coherence, so that we would know how to interpret the numbers (reliability coefficients) that we come up with.

So fear not the numbers. They are there to help us. But the interesting part is deciding what we want to measure, what’s the best way to do so, and how to interpret the results. No formula can do all that for you (Meehl and Goldberg notwithstanding).

Have a qualitatively and quantitatively good day!