Assessment Validation—Why We Miss Things Easily Found by Auditors
After an auditor has completed assessment validation and found a mountain of issues, you might wonder how you managed to miss them. You’re a competent trainer and assessor, so why does ASQA’s validation of your assessments differ so drastically from your own?
Let’s try a quick experiment to test your attention to detail. If you wear a watch, follow the instructions below.
Step 1
Place your hand over the face of your watch without looking at it. No peeking.
Step 2
Think about whether the watch has numbers all around the dial, or whether it has some numbers at 12, 3, 6, and 9, or no numbers at all. Think of every colour included on the watch, and any other notable details that it might have.
Step 3
Look at the watch to check what you remembered.
How did you go? Don’t be hard on yourself if you struggled. You look at your watch constantly throughout the day, but probably don’t pay much attention to the details, because your focus is narrowed to one thing: finding out the time.
The analogy is similar to what can happen in assessment—we focus on a desired outcome, develop an assessment tool to achieve the outcome, but don’t analyse the competency requirements for compliance. We’ve completed the task so many times, we’re blinded to the crucial details.
This automatic completion of tasks is what psychologists call unconscious competence—the final stage in the four stages of competence (below).
Unconscious competence usually saves us a ton of time. But when we develop an assessment tool while in unconscious competence mode, we slip into danger. As we mechanically map assessment items against units of competency, assuming that we’re working correctly because we’ve done it thousands of times, we blind ourselves to the intricacies of the unit. We fail to consider the details, and so we fail to write a compliant assessment. When it’s time for assessment validation to be completed by an auditor, they’ll find issues.
Creating an assessment that satisfies the required competencies is finicky. We can’t take anything for granted. When we’ve completed every question, and we’re ready to check that our assessments satisfy the competency requirements, we must adopt a manner of scrutiny, and follow this process for every question:
- Identify the key steps you’d take when completing the question. Write them down.
- Identify the Performance Criteria (PC), Performance Evidence (PE), and Knowledge Evidence (KE) from the unit of competency.
- Map the PC, PE, and KE to your key steps.
If you followed this process for a question, did you manage to map all criteria and evidence to the key steps for the task? You’ve done well if so. If not, you’ve identified what’s missing, and can update the question to include them. You’ve temporarily shifted back to conscious competence, and as a result, the stringent compliance requirements that are tested in assessment validation have a much better chance of being ticked off.
Auditors can’t afford to work in unconscious competence mode. To be successful, they must be meticulous, or risk missing the nitty gritty that reveals non-compliance. The same stance must be taken by trainers when developing assessments. Automatic mode just won’t cut it. By shifting back to conscious competence, trainers will be able to create high quality, compliant assessment that satisfies auditors, and properly tests students.
Written by John Price from the Vet Gurus
Originally published at https://www.axcelerate.com.au.
