How a Confusion Matrix Behaves Under Distributions of Prediction Scores

As algorithms increasingly make decisions about human affairs, it is important that these algorithms and the data they rely on be fair and unbiased. One of the diagnostics for algorithmic bias is the Confusion Matrix. The Confusion Matrix is a table that shows what kinds of errors are made in predictions. While everyone who works with data knows what a Confusion Matrix is, it is a more subtle matter to gain intuition for how it behaves under different kinds of distributions of predictions and outcomes and the range of possible decision thresholds.

In this article, I walk through an interactive…


NLP, Knowledge Graphs, and the Three Pillars of Intelligence

Jibo, Echo/Alexa, Google Home

The Three Pillars of Intelligence

To Amazon, the reception for its voice agent, Alexa, was a big surprise. Apple’s Siri had put voice input onto smartphones. But here was a new class of device that you could shout at across the kitchen to set a timer, play music, or look up facts on the internet. The culture adopted Alexa with an avalanche of jokes and memes. Google soon followed with its own smart speaker called Google Home. …


What will we need to know in the Age of Artificial Intelligence?

This article is a sidebar to Part I of a four-part series, How To Be A Robot Psychologist.

Here, we review the scope and depth of knowledge that humans have been required to master for our survival and well-being through the ages. The trend over time is that any individual, or even group of experts, has been able to comprehend and command relatively less and less about the expanding technology universe that civilization creates. The coming age of AI poses unprecedented challenges to our conception of how nature, technology, and mentally competent beings interact.

The chart below summarizes four domains…


This is Part IV of a four-part series: How to Be a Robot Psychologist

Part I: Why Robot Psychology?
Part II: Human and Robot Psychology and Cognition
Part III: How Do Conversational Agents Know So Much?
Part IV: Reverse-Engineering Conversational Agents

If you want to know who played Spock in the TV Series, Star Trek, then Alexa can tell you in two seconds. If you want to know who won the 1934 World Series, Alexa can tell you that. But there are limits to the questions Alexa, Google Home, Jibo, or Siri can answer. Part III of this series, How…


Jibo, Echo/Alexa, Google Home

This is Part III of a four-part series: How to Be a Robot Psychologist

Part I: Why Robot Psychology?
Part II: Human and Robot Psychology and Cognition
Part III: How Do Conversational Agents Know So Much?
Part IV: Reverse-Engineering Conversational Agents

The Three Pillars of Intelligence

To Amazon, the reception for its voice agent, Alexa, was a big surprise. Apple’s Siri had put voice input onto smartphones. But here was a new class of device that you could shout at across the kitchen to set a timer, play music, or look up facts on the internet. The culture adopted Alexa with an avalanche of jokes…


Cartoon of robot watering an indoor Christmas tree with a hose. Owner is upset.
Cartoon of robot watering an indoor Christmas tree with a hose. Owner is upset.

Series Abstract

Can you diagnose and fix your car? Can you reboot your router? We get along with technology better when we have some operational understanding of how it works. As we approach an age when robots and other artificially intelligent agents surround us, we must expand our mental models of machines to include those that display autonomous goal-directed behaviors and even evoke notions of intelligent awareness. We deal with other people and with our pets and other animals at a psychological level using our intuitive “Theory of Mind”. In the AI age, we must extend these skills to become robot psychologists


This is Part II of a four-part series: How to Be a Robot Psychologist

Part I: Why Robot Psychology?
Part II: Human and Robot Psychology and Cognition
Part III: How Do Conversational Agents Know So Much?
Part IV: Reverse-Engineering Conversational Agents

We get along better with technology when we understand it. In the coming Age of AI, we will be surrounded by autonomous AI agents that speak and listen in natural language, operate on their own volition, and generally seem to behave more like sentient creatures than rote machines. How can we understand these things, to help us manage?

In…


Math Helps Us Decide

How can we understand the Kavanaugh hearing and people’s reaction to it? The well-established logic of probabilities can help.

The testimony before the Senate Judiciary Committee left a lot of uncertainty and a lot of room for interpretation. Was Christine Blasey Ford really assaulted? Did Brett Kavanaugh do it? Is her testimony a sincere public service or a con job? Is his reaction authentic indignation or the rage of an entitled bully?

We can call on Bayes’ Rule to help us understand how we and others can come to different conclusions. Bayes’ Rule is a venerated tool in science for…

Eric Saund

Eric Saund is a researcher in Cognitive Science and a consultant in AI and Machine Learning.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store