QA Evaluations, And How They Aren’t

Why I can’t take you seriously…

The following is a dispute written by me as part of my workplace’s dispute process, contesting a quality assurance evaluation I got, and I’m posting it as an example of how benign the effect of evaluations like this are in improving actual customer service quality.

In case you don’t know, I’m a college graduate stuck in a low-skilled, low-income job, and sometimes I get frustrated with things that I know are wrong but am powerless to fix because, as goes the repeated evaluation, I am not the “right fit” for the myriad positions for which I have applied requiring any amount of critical thinking. As much as I’ve tried to refrain from this belief, it’s starting to appear to me that perhaps my problem is that I’m just not that White which, lately (although not necessarily in this specific case), is how things tend to end up here.

Enjoy.

The source of my confusion is pretty clear. The wiki states two things regarding individual purchases (i.e. purchases that are not a complete pair of glasses, which is defined as “Frame, lenses and lens options purchased in same transaction”):

1) Items purchased separately will be discounted 20% off of the retail price
2) Frames: Any frame available at provider locations: 35% off retail price

So, I’m still confused as to the price of, or discount on, the frames when not purchased as part of a complete pair of glasses. Does one of these discounts trump the other? If so, in what circumstances? Is the 35% discount only available at certain provider locations? Are lenses 20% off and frames 35% off, no matter what? If so, why is “items purchased separately” not corrected to read as a “lens-only purchase”?

Nowhere is this clarified, to include the actual evaluation recommendations. If the point of QA is to improve customer service quality, I can’t say it’s helped.

The evaluation coaches that the the wiki advises “if they JUST need lenses, then the separate items will be discounted 20% off”, but the reason for my deduction in points is listed as being “confused as to what the cost for frames would be”. So, now I’m additionally confused as to whether I was deducted for not knowing about lenses or not knowing about frames. In any case, the wiki discounts as listed above have still not been clarified.

The evaluation goes into no detail for the other deductions on points. It is suggested that I either did not use the appropriate tone, did not speak clearly, did not avoid slang, did not use correct grammar, did not employ conversational skill and/or build rapport, or did not use voice inflection appropriately; however none of this is explicitly coached, except to say that I was confused. Which I was and remain.

The evaluation also goes into no detail for the deductions in the section relating to using the “appropriate contact handling procedures, appropriate use of the customer’s name, or following transfer/hold rules”. The only coaching that I can see that relates to this is the recommendation to ask the member to refer to the provider, however the evaluation states that I did this, and I did. Was there some other reason for the deduction in points; if so, what was it?

The only explained point loss is in the section that requires I “demonstrate knowledge about products and services, have a solid understanding of call center policies and procedures”. The coaching did not suggest that I don’t have a solid understanding of call center policies and procedures, but simply that I was confused by the wording of the wiki. It’s true that I was unable to demonstrate knowledge about the product/service, and the evaluation has not coached on that at all, so there has so far not been any improvement on this. My knowledge is as extensive as the clarity of the information presented in the wiki. As for call center policies and procedures, I can only surmise that this refers to the fact that I should have referred the member to the provider, but as was explored above, the evaluation says that I did do this.

These evaluations are nearly completely subjective. Not just this one, but all of them. There are no metrics, there is no indication as to how many points will be taken off for any given infraction, and in this case, there is no clarification as to what the right answer would have been or should be.

Because there is no verifiable numerical component to this data, no one is able to build statistics out of this information, which by definition means that there is no actual measurement being done. There is no defined assignment of point value for each of the criteria, and there is also no standard of method providing a way to determine whether one evaluator would take more or fewer points off for any given infraction, making the evaluations completely susceptible to subjective inconsistency. I’m just submitting this dispute for comment, not because I expect the score to be changed or the QA system to be evaluated. Certainly, I could listen to the call myself and try to estimate which of these criteria I did not fill, as well as attempt to approximate how many points those infractions are appropriately worth, but that would undermine the point of having a system in the first place.

Aside from how this affects my personal eligibility for a raise, there doesn’t seem to be any real reason for these evaluations when they provide no clarification on the original issue and also offer no objective standard of measurement. Not only does this diminish the impact of any negative QA, but also belittles the importance of positive QA scores, as well. I don’t know who these disputes get sent to, but I hope someone who has some input on how these evaluations are structured and conducted is getting feedback from agents. If not, I’m happy to discuss in a meeting any way of introducing a more verifiable standard of agent evaluation.