How To Prepare Your Workforce to Think Like AI Pros

If you felt a sudden urge to smile when you saw this rock, you’re in good company.

As humans, we often irrationally describe human-like behaviors to objects with some, but not all, characteristics (also known as anthropomorphism) — and we’re seeing this occur more and more with AI.

In some instances, anthropomorphism looks like saying please and thank you when interacting with a chat bot or praising GenAI when the output generated matches your expectations.

But etiquette aside, the real challenge here is when you see it ‘reason’ with a simple task, like summarizing this article, and then expect it to effectively perform the same on an anthology of complex scientific articles. Or, when you see a model generate an answer about Microsoft’s recent earnings call and expect it to perform market research by providing the model with the same earnings transcripts of 10 other companies.

These seemingly similar tasks are actually very different for models because, as Cassie Kozyrkov puts it, “AI is as creative as a paintbrush.”

The biggest barrier to productivity with AI is human’s ability to use it as a tool.

Anecdotally we’ve already heard of clients who rolled-out Microsoft Copilot licenses and then scaled back the number of seats because individuals didn’t feel like it added value.

Chances are that those users had a mismatch of expectations between the problems AI is well-suited to solve and reality. And of course, the polished demos look magical, but AI isn’t magic. I’m very familiar with the disappointment felt after the first time you realize ‘oh, AI isn’t good for that’.

But instead of throwing up your hands and quitting GenAI, you can work on building the right intuition to more effectively understand AI/ML and avoid the pitfalls of anthropomorphism.

This article was originally published on VentureBeat.

Defining Intelligence and Reasoning for Machine Learning

We’ve always had a poor definition of intelligence. When a dog begs for treats, is that intelligent? What about when a monkey uses a tool? Is it intelligent that we intuitively know to move our hands away from heat? When computers do these same things, does that make them intelligent?

I used to be (all 12 months ago) in the camp that was against conceding that LLMs could ‘reason’.

However, in a recent discussion with a few trusted AI founders, we hypothesized a potential solution: a rubric to describe levels of reasoning.

Much like we have rubrics for reading comprehension or quantitative reasoning, what if we could introduce an AI equivalent? This could be a powerful tool used to communicate to stakeholders an expected level of ‘reasoning’ from an LLM-powered solution, along with examples of what is not realistic.

Humans Form Unrealistic Expectations of AI

We tend to be more forgiving of human mistakes. In fact, self-driving cars are statistically safer than humans. Yet when accidents happen, there’s an uproar.

This exasperates the disappointment when AI solutions fail to perform a task you might have expected a human to perform.

I hear a lot of anecdotal descriptions of AI solutions as a massive army of ‘interns’. And yet, machines still fail in ways that humans don’t, while far surpassing them at other tasks.

Knowing this, it’s not surprising that we’re seeing fewer than 10% of organizations successfully develop and deploy GenAI projects. Other factors like misalignment with business values and unexpectedly costly data curation efforts are only compounding the challenges that businesses face with AI projects.

One of the keys to combating these challenges and unlocking project success is to equip AI users with better intuition on when and how to use AI.

Using AI Training To Build Intuition

Training is the key to coping with the rapid evolution of AI and redefining our understanding of ML intelligence. AI training can sound pretty vague on its own, but I’ve found that separating it into three different buckets has been useful for most businesses.

1. Safety. How to use AI safely and steer clear of new and AI-improved phishing scams.
2. Literacy. Understanding what AI is, what to expect of it and how it might break.
3. Readiness. Knowing how to skillfully (and efficiently) leverage AI-powered tools to accomplish work at a higher quality.

Protecting your team from AI’s potential pitfalls (AI safety training) is like wrapping a new cyclist in bubble wrap: it might prevent scrapes and bruises but won’t prepare them for the challenges of mountain biking.

Meanwhile, AI readiness prepares your team to use AI-infused tools to their fullest potential. This is usually the source of discrepancy between expected performance and reality.

The more you give your workforce the chance to safely interact with GenAI tools, the more they will build the right intuition for success.

We can only guess what capabilities will be available in the next 12 months but being able to tie them back to the same rubric (reasoning levels) and knowing what to expect as a result can only better prepare your workforce to succeed.

Know when to say, ‘I don’t know’, know when to ask for help, and most importantly know when a problem is out of scope for a given AI tool.

Let’s Connect

If you found this article valuable, I’d love to connect further! Feel free to subscribe to my content right here on Medium, connect with me on LinkedIn, or contact the Further team to discuss upcoming projects and opportunities.