My Beef with the QS World University Rankings Methodology
Maybe someone who is better versed in statistics can explain this to me. Maybe I’m just absolutely wrong and I need to be told why these things are the way they are. But the recently published list of university rankings by QS World University Rankings seems absolutely absurd.
Put aside for a moment, if you will, the even larger absurdity that I even have to write this post to begin with. There’s a certain amount right and a certain amount wrong with college ranking culture. But that’s another post and another time. This is about the fact that people will point at this list as a reason to think about specific universities in specific ways, and for many reasons, that might influence a decision in an incorrect manner. And especially considering how expensive college is, and how many schools are effected by lists like this (read: every school), it seems like something that should be mentioned.
I opened the list, and was greeted by the typical slew of Ivy Leagues and prominent international universities. Okay. That makes sense. But a few schools started to pique my intuition that all was not as it seemed on the list. Namely, the placement of the UC schools. Now, I am not bashing against the University of California. I wouldn’t bash against any school. But in terms of perception and renown, it doesn’t seem to make sense that UC-San Diego and UC-Davis would rank considerably above schools like Washington University in Saint Louis and Dartmouth College. Comparing QS to the US News & World Report Rankings (which I’ve certainly got beef with as well), Dartmouth clocks in at #11, WU-SL at #14, with UC-Davis at #38 and UC-San Diego at #37. Of course, there are going to be discrepancies between the lists, but that seems like an awfully large one, no? So, of course, I cracked the methodology open and found a few problems I’d like to point out. Please note, I am not a statistican or a data scientist, but listen to the argument, not the voice saying it.
1: Why is Employer Reputation only given 10%? Given the fact that college degrees now-a-days are at least viewed as a ticket to employment, why are we weighting them this lowly? Even if you disagree with the fact that people should view college in a different manner, doesn’t mean that that affects the reality of the situation. Surely there must be something more significant coming in the rankings, or more logical spaces with which to use that weighting right? Nope.
2: Student to Faculty Ratio at 20%?! They say that you can’t really find an objective measure of teaching quality, so this is what they substituted. Huh? These, on face, appear to measure two completely different things. A small class size has no bearing on the quality of the class at all, but this insists a correlation. Furthermore, what is the definition of “academic stafff”? Do TAs and student teachers count? Surely a class taught by a brilliant Nobel Laureate in his field can’t be weighted for less than a class of twenty taught by a professor who merely scrapes the surface of his subject? Not to mention the fact that their best stab at an objective measurement of teaching quality is DOUBLE the size of an actual objective measure that people would look on more favorably being weighted higher (Employer Reputation). So why on on earth is this twice the size of that?
3: International Faculty and Student Ratio at 10% (five percent for each). Now I completely get it. Diversity is important not only for equal opportunity, but also for getting more perspectives integrated into a learning environment. There are plenty of reasons why diversity is something to be considered. But when it has exactly the same weight at how employers look at a university, that’s inane. The two simply cannot be equated, and there is a much higher chance of a correlation between the quality of a school and how employers look at it, then the percentage of students that come from other countries to attend that school. This also means that universities outside of the United States have a dramatic advantage, due to the logistics of moving between countries in the EU (again, not as difficult as US visas etc).
Look, it’s bad enough that we have to deal with college rankings as a means for making a decision about what school is right for us. It’s like trying to pick which girl you should dance with based on who your friends think is the cutest. It kinda points you in the right direction, but can make you seriously unhappy in the long run. But to have your friends shouting in your ear that the girl who is most likely to give you a swirly when you have your back turned is also one who you should look at more favorably? Well we just shouldn’t do that to students, should we?