A puzzle and an explanation
First the explanation
In October of last year I posted a few keen and hopeful blog posts concerning my Master’s studies and my intention of documenting the reflections that it might give rise to. I received some interesting and encouraging feedback and then, in my classic style, abandoned it. At some point I even unlisted them, but I made them visible again now because it seems even worse to hide my failed attempt at constancy. I believe the above merits a brief but honest explanation: I abandoned the endeavor because I realized I did not like the course of study I was embarked on. I didn’t really feel like I understood what sociology was about, and the bits I understood I simply found unappealing and boring. So what was I to write about, if I sat in lectures feeling completely blank and devoid of questions? What was I to reflect upon, if I read the papers and wrote my essays dutifully but without love, sometimes bored to death of all of it? Plus, I had more pressing worries than a blog post, like finding a research topic amid all this material that seemed so sterile.
Hope is the last thing you lose
I still don’t really know what sociology is; it feels like the more I learn the more I can only conclude that sociology can be defined as “whatever sociologists at that point in time are researching and publishing about in their journals”. But I did find a thesis topic, and I’m confident that I will manage to muddle through the course despite my lack of passion for it. And now that I am gradually working towards writing my thesis, I feel like I finally have some reflections to offer on the process, something to broadcast to this void called the Internet and whoever may be interested.
My topic (as submitted provisionally to the graduate studies administrator and hopefully to become more appealing and specific) is “Assessment of the health impacts of volunteering in the United Kingdom”. At this point in time I am simultaneously looking at literature on the volunteering-health relationship and also exploring the data that I will be using. Understanding Society is a large, nationally representative and longitudinal survey of the UK that asks about volunteering in 2 of its waves.
Finally, the puzzle
Today I want to talk about something strange that I found in one of the papers that I was going through, and see if anyone has insight about it they would like to contribute which might clear my understanding. You can see the actual paper here. It seems worrying to me that something like this can get published in an important peer-reviewed journal; it doesn’t speak too well about social science. To summarize, they set up a two-step multiple regression model for the relationship between volunteering and both subjective wellbeing (SWB) as well a measure of “neighborhood well-being”. The authors include a series of control variables in their model, mainly personality traits, and these are the results they get:
(Note: I just realised, they also fail to include the meaning of the */** in this table. But in the previous one it says “*p < .05. **p < .01”.)
The weird thing about the table, as I see it, is the sign of the regression coefficient of volunteering on PWB: -0.08. The authors constantly talk about this well-documented, positive link between volunteering and PWB in the literature. So, am I missing something here? I don’t think so: the directions of the other coefficients point towards what you would expect, from a common sense perspective, i.e. depression decreases your subjective well-being, optimism increases it. So then, the model points towards volunteering, also, decreasing your PWB? It seems counter-intuitive, but perhaps there could be an explanation, and the effect size does not seem particularly large anyways. But the weird thing is that the authors simply fail to mention this in their discussion. All they say about the results of their regression is the following:
The first analysis indicated that volunteering is a significant predictor of PWB (β = –.08, p < .01). It contributed an additional 0.6% (∆R2= .006, p < .01) of the variance in PWB, even when contributions of other factors such as personality traits and psychological resources were accounted for. (Mellor et al., 2009, p. 151)
They use this in their discussion to say that “volunteering accounted for a significant amount of the variance” (Mellor et al., 2009, p. 153) and then go on to discuss the protective effect of volunteering, its role as a “benign condition”, and so on and so forth. As far as I could tell, the authors fail to consider both the substantive relevance of the “statistically significant” effect and its direction. All they say is that it contributes 0.6% of the explanation of the variance. That’s it. Is this a sufficient level of analysis? Did I just miss some vital piece of data or explanation in the text? Summing up this whole rant: I am impressed that something with a gap like this can get published. Of course I had heard about mediocre and shitty papers being published in peer-reviewed journals, I guess I had just never noticed it first-hand. That’s all I wanted to share with the world for now.
Mellor, D., Hayashi, Y., Stokes, M., Firth, L., Lake, L., Staples, M., and Cummins, R. (2009). Volunteering and Its Relationship With Personal and Neighborhood Well-Being. Nonprofit and Voluntary Sector Quarterly, 38(1), 144–159. http://doi.org/10.1177/0899764008317971