How Does “Soft Science” “Get Hard”?
I am sitting in a graduate-level clinical science seminar at a public university in Southern California. The professor has a Ph.D in Clinical Psychology from a prestigious university. It is our third week of class and we are learning about clinical research methods. I am working on a master’s degree in psychological research and learn in this class how thoroughly disturbed I am by this discipline.
The professor is describing a research project she conducted during her doctoral training that led to numerous follow-up studies and remains one of her primary ongoing research projects.
The original project’s aim was to develop an effective treatment for childhood anxiety. Its sample was comprised of anxious children from two different elementary schools who were around the same age and varied in terms of ethnicity and gender.
What she found was that Latinx children were more responsive to the therapy at one elementary school but not the other. This led to numerous efforts to determine what it was about the Latinx children that made the intervention more effective. Was it some other factor that they hadn’t measured that was doing it? Was it the particular sample of Latinx children? She described all of the statistical analyses and follow-up studies that came after this one to try to figure this out. Her description was indeed a helpful overview of clinical research methods, but it was also a performance that sticks with me for other reasons.
She is describing this project and its findings with increasing passion and grows excitedly erect in her chair when she gets to the statistics. She is gesticulating around her midsection and fervently animating the research process. My brows are serious as her enthusiasm builds because I am growing increasingly unsettled by something.
I am watching her demonstrate objectivity while wondering about the ethnicities of the therapists who administered the treatment to the children. If the therapist and child were of the same ethnicity, then they would probably feel more comfortable and therefore be more receptive to the treatment. It made total sense to me that a Latinx child would feel more comfortable with another Latinx therapist, which would explain why the treatment was more effective for Latinx children than the children of other ethnicities. I grow more agitated the longer she describes the statistical analyses. Could she really have ignored this glaringly obvious feature of the research design?
After she finishes speaking I raise my hand and ask: “Do you know the ethnicities of the therapists?” Her eyebrows raise, eyes widen, and arms fold quickly as she leans back in her chair and says “Oh, you’re…” Her tone flattens and she says with snapped head-nods that the therapists in San Diego were Latinx, and that she and her team were “considering this as a factor.” She says that they do not keep a record of the ethnicities of the therapists as I nod solemnly. I feel shut down and attacked, but that’s probably just because I’m freshly traumatized by other forms of epistemic violence; what I realize now is that I made a damn good point that undermined both her original study and every subsequent effort she has made since then to tailor anxiety treatments to Latinx children. Her gaze snaps quickly toward the floor when we pass by one another in the hallway the following semester.
(I got an A in the class, in case you’re wondering. I actually published my final paper for this class on Medium— check it out here.)
What is going on in psychology right now? You’ll answer this question differently depending on who you are, and so I’ll answer it as who I am: there’s some phallic nonsense going on in psychology right now that would probably make Freud choke on his cigar.
Statistics is a very useful tool that can provide fascinating insight into the human condition. There are a variety of measures that are used in statistics. Some are central conceptual variables like α (alpha), which refers to the point of statistical significance, and there are others like Cohen’s d that white men named after themselves.
Cohen’s d is commonly used in psychology because it is a measure of effect size. Effect size in psychology refers to a quantification of the impact the treatment (or, experiment) had on participants. You obtain Cohen’s d by finding out the difference between how you measured the groups of participants in your study, and then dividing this difference by the standard deviation of your dataset, which is a measure of the variation among all of the measurements you collected.
If your measurements are significantly different from one another, then Cohen’s d is said to be either large, medium, or small depending on how much of an effect your experiment (or treatment!) had on the (willing?) participants. In the case of the research I described previously, Cohen’s d was largest for the Latinx children because they were the ones who were most affected by the treatment in comparison to the other groups of ethnically-categorized anxious children.
I once led a review session for a midterm exam in a statistics class, and while finishing up a particularly lengthy problem on the whiteboard, I kept my back to the class and asked, “And finally, the question that’s probably on all of our minds: how big is Cohen’s d?” There were a few laughs (or maybe just one?) as I calculated the d. I found that it was of medium size and drew a box around it to indicate that it was the final answer. I carried on with the rest of the review pretending like there was nothing funny going on.
There is another statistical measure that is called p or the p-value. It represents the percent chance that the measured effect happened randomly and not because of the experiment (or treatment!). Minimizing p is the goal, as the smaller the p, the more significant the finding.
If a psychologist finds a small p, then this means that there is a very small chance that the measured differences occurred randomly and not because of whatever happened during the experiment (or treatment!). This allows the psychologist to then not only claim but also feel significance.
This is why p is also called the significance value: the significance of the finding increases the smaller p becomes. This means that at the end of all this hard work it is ultimately the p that determines signifiance.
This said, now let’s bring back Cohen’s d.
Given that Cohen’s d indicates effect size, the bigger the effect size, the smaller the chance that what happened was just because of randommness. Given that p represents the chance of randomness, it therefore follows that Cohen’s d and p have an inverse relationship. In other words:
The bigger the d, the smaller the p.
If a psychologist finds significance, it’s because the d got bigger and the p got smaller. This is a very important statistical relationship to remember when both conducting and understanding the methods of psychological research.
I am, of course, being snarky; there are plenty of other measures and analyses out there besides Cohen’s d (e.g., Pearson’s r, the t, etc.).
Seriously, though — let’s go there. If you laughed at the big d small p comment, why was it funny? C’mon, don’t be shy — say it!
The bigger the dick, the smaller the pussy. Yes, let’s go there.
Don’t stop now, let’s keep going: isn’t this what makes insertive phallic pleasure most pleasurable, a big dick and small pussy? Small pussy, tighter hole, and more pleasurable insertion for ds like Cohen and anyone else who likes getting large and fucking around! It’s vulgar, it’s nasty, it’s insight-filled!
C’mon don’t go yet there’s more here: to what extent does this reflect how sexism is performed this time of NastyWomen, Fake News, and phallic embodiment?
“Bigger the dick, smaller the pussy” reflects those manly men getting louder to silence the scared and terribly sensitive woman.
“Bigger the dick, smaller the pussy” reflects those gay men who won’t fuck femmes.
“Bigger the dick, smaller the pussy” reflects phallic affirmation of the masculine at the violent expense of the feminine.
“Bigger the dick, smaller the pussy” chant the scared men who aim to eliminate the feminine.
“Bigger the dick, smaller the pussy” reflects whenever the feminine is disregarded as a valid and important site of lived experience that deserves to be honored and respected regardless of the genital genesis of our gendered significations.
Or maybe it’s just a cooincidence, who knows?
All I know is that it’s a fucking tragedy watching soft science get hard.