“Even the Smartest People Will Make Bad Decisions with Bad Data”
Kenny Mendes, a guy who runs a recruiting at a software startup, dedicated two entire years of his life to figuring out how to hire better employees and encourage existing ones to perform at their best using mountains of data and AI algorithms. It was his white whale, his dream. But after two years he learned some hard lessons. (quotes from The Wall Street Journal)
The experience led him to believe the problem is too complex for the current generation of software. The limitations of current approaches, he says, boil down to the difficulty of drawing valid conclusions from incomplete data.
What kind of incomplete data is he talking about?
Measurements of employee performance at any given company are based on the set of people hired and lack information about candidates who were passed over — or weren’t even interviewed — who may have, say, produced more in less time.
Okay, but surely you can still draw some important conclusions based on the employees (and data) you already have? Well, sadly, the problems don’t end there.
Moreover, management systems can’t account for conditions outside the office that may energize or depress individual employees at work — especially personal conditions that can shift unpredictably. On top of that, human psychology is a wild card; if workers know their overseer is tracking hours on the job rather than output quality, they may spend an extra hour a day at the office simply chatting by the water cooler.
So what does this all mean? Mendes summarizes it this way:
Even the smartest people will make bad decisions with bad data, and I think we have a lot of bad data in this process
In his own way, Mendes is encouraging us all to be realistic about the quality of our data and, therefore, the quality of the decisions produced from it. Sometimes, even after two years of work and tons of data, you’re not going to get the predictive results you want. Rose colored glasses aren’t going to serve you any better in the world of AI than in any other field.
If you’d like to read an AI success story, check out what we accomplished by throwing an entrepreneur, a data analyst, and a CTO in a room for three hours.
Note: I am 95% sure the guy pictured above is not Kenny Mendes, but hey, I’m just as fallible as AI, so who knows.
