Stupid exam or stupid doctors?

A year ago when I sat the Key Feature Problem (KFP) paper as part of the RACGP exams, it felt like a big deal. Still, I would have been pretty surprised if I were told the exam would make headlines in national newspapers.

But with a fail rate of more than 50%, that’s exactly where the 2015 KFP exam ended up.

Sure, it has a low pass rate, but there are many such exams in other fields. So why did this one get attention?

The thing is, passing the exams suggests the candidate has demonstrated competency. And by implication of a seemingly cruel binary, failing the exam suggests the candidate was not competent.

And no one wants to see an incompetent doctor. That’s why the story has purchase. That’s why it made headlines.

Sydney Morning Herald: GP exams — More than half of candidates fail

This isn’t to say the doctors who fail the KFP are incompetent; far from it. In fact, the popular sentiment is completely the reverse.

It’s a striking cognitive dissonance in the minds of trainees, doctors and news reading patients: trainee doctors are highly capable, so what does it mean when more than half of them fail their final exam?

As has often been said about exams with high fail rates: either the flaw is in the exam itself, or the candidates.

And if the candidate is responsible, then it is either a deficiency in their training, or their study efforts, or more harshly, their aptitude.

Their aptitude may encompass not just knowledge and understanding, but also communication skills and their exam technique. I don’t think I’m missing anything here.

Of course it’s arguable that these pass rates simply reflect the exceptional standards expected of GPs. However, over the years, the pass rates for the KFP exam have varied wildly between 60% and 90%.

And unless there has been some radical, deliberate and secretive raising of the bar within the college, there is a deeper problem.

The RACGP suggests the problem is with the candidates, stating in a recent newsletter that it has “limited, if any, control over the selection of candidates who gain entry into the Australian General Practice Training, nor over the learning and exam preparation of non-training pathway candidates”.

These non-training pathway candidates include — in its own words — international medical graduates and a small percentage of ‘grandfathered’ GPs. To skip the PC line, none of these assertions are, in and of themselves, absurd.

The problem however, is that they meekly point in the vague direction of an explanation, rather than articulate it specifically.

And with this vagueness, the inference and analysis ends up pointing to individual candidates.

Keeping everyone in the dark, and giving no guidance, is unhelpful to everyone. It’s unhelpful to the local graduate. It’s unhelpful to overseas-trained candidates. It’s unhelpful to those in or out of training programs.

So why exactly do candidates find the KFP so problematic?

As the college won’t release the papers, and won’t permit the disclosure of past questions, we are left to discuss this by vague anecdote.

So here goes, anecdotally. As someone who did fairly well in the KFP, my biggest issue wasn’t the difficulty per se of the questions; it was the strangeness of them.

It contains the kind of questions you read thrice and ask yourself: “What do you want from me?”

That’s not necessarily a bad question. Day-to-day general practice has been known to occasionally spark such visceral reactions — it’s good for the soul.Thankfully, our patients are the ones who, if we allow ourselves to engage with them, help us answer these questions and find direction.

Sadly, a non-interactive exam question does not help resolve these abstractions.

This isn’t a surprise; there are natural limitations to the written examination of a practical field. Every translation will be imperfect. But with pass rates fluctuating by 30 percentage points, ought it be questioned and improved?

I honestly do not think that candidates are, in the true sense, complaining that the exams are “too hard”. Accusing them of being flawed is a completely separate charge. It’s not an issue of difficulty, rather one of consistency.

In any case, the question is pretty clear: with current fail rates of up to 50% in the KFP, either the exam is flawed, or exam candidates are undertrained, lazy or stupid.

Although more than one answer may be correct, which of the above is most likely correct?

Frankly, my money is on the exam.

Medical Doctor / Health Commentator

Medical Doctor / Health Commentator