Member preview
Psst...there’s an audio version of this story. Upgrade to listen.

How Your Brain Decides What You’re Seeing

Photo: Sean Brown via Unsplash.com

Your brain makes most of its decisions without bothering to consult with you. Some are trivial: If you had to decide which foot to put down first each and every time you started walking somewhere, you’d be mentally exhausted. So your brain (and body) decide for you, and off you go.

Some of these decisions are not so trivial. You’re moving through some long grass in India when you suddenly freeze. Your brain gibbers the word “tiger,” and then starts the internal alarm: “Awoogah! Awoogah! Awoogah!” It does that after you freeze. Your brain decided it saw a tiger because it added up all the bits of evidence from your eyes and ears and decided: That’s a tiger. And it made you stop to do something about it.

Many decisions require you to weigh the evidence for different options before you can make a choice. Is that a tiger or isn’t it? Do I want the burger or the hot dog? Is that a canoe in your pocket or another item of fiberglass sporting equipment? Your brain seems to solve this beautifully and silently: Without you knowing it, your neurons add up evidence, work out the best option, and tell you how to act.

How do we know this? In the lab, we’re not allowed to lock you in with a live tiger and some long grass. (“Did you see the tiger?” “No, I’ve always had half my arm missing, genius.”) Instead, we show you some dots. Not just any old dots — moving dots. We show you some moving dots and ask: Which direction are they moving in?

This is harder than it sounds. We can control the number of dots that move in one direction, and the rest move at random. When many dots move in one direction, the decision is easy:

Which direction are the dots moving? Here, about 30% of the dots are moving in the same direction. Easy, huh? Credit: Newsome Lab, Stanford University (http://monkeybiz.stanford.edu/research.html)

When only a handful out of thousands of dots are moving in one direction, the decision is very hard:

Which direction are the dots moving? Here, only about 5% of the dots are moving in the same direction. Not so easy now, huh? Credit: Newsome Lab, Stanford University (http://monkeybiz.stanford.edu/research.html)

We show moving dots because we can most easily study decisions about what you’re looking at. And when we show animals these dots, we know where the neurons that give a damn about moving dots reside, and we can then record their activity. (You’ll be pleased to learn that, no, most of your brain does not care about moving dots. It finds them just as boring as you do.) The animal sits and stares at the dots, and when it reaches a decision, it looks in the direction in which it thinks the dots were moving. If the animal gets it right, juice time!

When we record single neurons from various bits of brain (especially the cortex), we find a startling thing. Neurons accumulate evidence. As the dots move, and as the animal stares, single neurons increase their activity—but only if the movement is in the direction they like; if not, the single neurons sulk and go quiet. Some neurons like dots that move left; some like dots that move right; some like up; some like down; some like dots moving at 187 degrees. (Like people, some neurons just have to be different.) And the more difficult it is to see which direction the dots are moving, the slower the neurons increase their activity; when there is less evidence on which to base a decision, the neurons that tick off that evidence as it comes in move more slowly. Once some of the neurons’ activity is high enough, when it reaches a threshold, the decision is suddenly made: The animal looks in the direction that those neurons like. It decides: The dots are moving that way.

Why is this a decision? How do we know that these neurons were “deciding” and not just counting sheep out of boredom as the time passed?

Two reasons. First, we can see what causes errors. When animals look in the wrong direction, we can see the neurons that like this direction reached the threshold first. They signaled, adamantly but incorrectly, that the dots were moving in their favorite direction (and they’ll deck anyone who says otherwise). This happens because if only a few of the dots were moving in one direction, the neurons sometimes mistakenly add up the random movement as being in their favorite direction. So we know the decision depends on which specific neurons are the most active.

Second, we can force the brain to make a decision when there is nothing to decide. Sometimes we cheat. We make all the dots move at random and ask what direction they are moving in anyway. While this is cheating, it’s also clever: When forced to make a decision based on nothing, we can check that the neurons really were representing the decision and not just counting sheep or thinking about next Tuesday’s meeting with your boss about the copier incident. And they really do represent the decision: When the dots move at random, the animal always moves its eyes in the direction favored by the neurons that reached the threshold first.

(We can also force the brain to make a decision even when it doesn’t want to. We can artificially stimulate a group of neurons that all have the same favorite direction, and the animal will more likely make the decision in that direction — even if the dots were not moving in that direction in the first place.)

Your brain has special juror neurons that weigh up evidence and deliver their verdict: Mate, you were looking at some dots. Thanks, brain.

Where are these neurons? Much research has focused on a tiny parcel of the cortex named the lateral intraparietal area, or area LIP to its friends. Here we see lots of juror neurons weighing up evidence about dots. But very recent evidence suggests that this area LIP cannot be the sole decision maker. When researchers temporarily turned it off, nothing happened — the animals went right on making the same, correct decisions about which directions the dots were moving in. (Being good scientists, they also checked that they really had turned off this area LIP: When they asked the animals instead to remember where a light had flashed, they couldn’t do it when area LIP was turned off.) So while this area might contribute to making a decision, it seems unlikely to be the foreman delivering the jurors’ verdict.

But area LIP is not the only place we find juror neurons. Neurons that add up evidence have been recorded in bits of prefrontal cortex and in the regions below the cortex, especially in the massive, quiet striatum. There’s also a strong possibility that the threshold for a decision is not set in the cortex at all, but is instead set much lower down in the brain. It’s unlikely we have one brain region acting as a jury; rather, that jury is spread out over a range of brain regions and delivers their verdict by pulling together.

Wherever it happens, it turns out your brain might be set up to make decisions in the best possible way. The optimal way — or close to it. Our best research says that neurons do not add up their evidence for their preferred option separately, that they are not racing each other along separate lanes to the finish line. No, they are not well-disciplined 100-meter sprinters. They’re long-distance runners, pushing and jostling and elbowing each other for supremacy. Every bit of evidence that increases the activity of one neuron decreases the activity of other neurons; evidence for one option is used as evidence against all other options. Which is the optimal way to decide, because it squeezes every drop of information from every bit of evidence: not just that it favors one option, but that it also counts against the other options. And that’s exactly what neurons seem to do.

Another clue that your brain might be set up to make decisions in the best possible way is that it can move the threshold for a decision. By lowering your threshold for a decision, you need less evidence to reach the threshold, and you decide faster. But you also make more errors. If you can take your time, you can raise the threshold: You can wait for more evidence before making a decision. Sure, it takes longer, but you make fewer errors. But if you wait too long, you’ll never make a choice. So, what is the best threshold?

It seems your brain can learn the best threshold. For a particular decision, it can learn the best trade-off between waiting longer and deciding now. And it does this by aiming to maximize the reward you get in the long run. For the dot-motion task, this is getting as much juice as possible by taking just long enough to make the correct choice most of the time, but not waiting so long that you are absolutely, positively, 100 percent sure of the direction — and getting only one drink of juice an hour.

All these clues point to the idea that brains make decisions about evidence by using Bayes’ theorem (or something close to it). In other words, that they use the optimal method in probability theory for making a decision between alternatives, given noisy information. Your brain does statistics, even if you think you can’t.

This is not the whole story of decision-making. We have focused here on how your brain decides what it sees. There are whole other types of decision-making. For example, deciding what to do next (we’ll cover that in a future piece). Or deciding the best route to get home through traffic. Or indecision.

What is indecision? When you just don’t care enough. Or too much. You can accumulate as much evidence as you want yet can’t make a decision. Dr. Antonio Damasio has described a patient, Elliot, with extensive damage to a tiny portion of his prefrontal cortex (the ventromedial part, if you’re interested) as a result of surgery to remove a brain tumor. Elliot experienced absurdly extreme indecision. He accumulated endless evidence but could not make a decision. For example, when asked to make a new appointment to meet Damasio, Elliot would sit for hours with his diary, weighing the pros and cons of making the appointment on next Tuesday or next Thursday. Endlessly accruing evidence yet never making this seemingly trivial decision.

Damasio contends this is because the patient could not access his emotions: that in damaging this tiny part of cortex, the link between the rational decision-maker and irrational emotion centers was missing. As a result, there was no point at which the threshold was reached: because the sensation of being right, the feeling that you’ve got the right decision, was missing.

And that Elliot couldn’t decide means the words “rational” and “irrational” are precisely the wrong ones to use. It is not rational to spend an infinite amount of time making a decision. As you sat completely still deciding whether to book a hair appointment next Tuesday or next Thursday, a predator would eat you. Or you’d starve to death. No, there has to be some other decision-maker that takes charge when the slow jurors are unable to come to a verdict. Something that says: “I’m choosing the bright-pink shoes, and there isn’t a goddamn thing you can do about it.” Emotions are not irrational; they are crucial to rapid, life-saving decision-making (and impulse purchases).

Your brain is a society of decision-makers. Each, on its own, is trying to make the best possible decision for you, right now. One, as we’ve seen, is a jury, studiously adding up evidence and weighing alternative options until a verdict is delivered. Another just wants to the do what feels right, what feels good. Together, this society makes you a formidable team.