How to find what you weren’t looking for

Many products have a problem with ‘discovery’. But the answer could come from a surprising place: Chinese philosophy.

Flick right. Flick right. Flick right.

I’m flicking through Netflix. Through the shows and movies Netflix recommends for me to watch. Top picks for Adam. Each item on the list is like something I’ve watched before. But I don’t want to watch them. 15 minutes later, I’m re-watching a series I’ve watched dozens of times.

I like the ‘Discover’ playlist on Spotify, but it has the wrong name. It’s at best agreeable, at worst…meh. But I haven’t discovered any new artists there. I discover new artists from recommendations, or by nosing around related artists, or overhearing music in shops. Almost random.

Pinterest sends me food fads, workout routines and basketball pics.

Instagram sends me basketball pics, memes and food fads.

None of it is wrong. But none of it is interesting.

These products treat ‘discovery’ as showing me more of the same. Discovery is vital for the success of content-based products. But it seems that even these advanced businesses don’t understand it.

There’s something wrong with ‘discovery’.

Pattern Problems

Last week, I watched author and academic Prof. Michael Puett lay out his interpretations of Chinese Philosophy, as outlined in his book The Path.

Challenging the idea of an ‘authentic self’, his interpretation of Chinese philosophy suggests that our belief in a stable ‘self’ is deceptive.

(Go with me on this).

In fact, what we consider to be ‘personality’ are actually patterns of perception and behaviour. Patterns that we develop from a young age — that we get stuck in, because we’re not aware of them. Patterns that come to dominate every aspect of our thoughts, feelings and decisions.

This is us.

Humans are social creatures. And so other people trigger or ‘pull out’ these patterns from us.

Consider the frustrating, repetitive interactions you have with a problem colleague or parent. Consider the friend who you see do the same thing in relationships, time and again.

The problem isn’t anything to do with either one of you, your personalities.

These are our patterns at work.

This is an ancient idea. But it’s one that today’s psychological research provides supporting evidence for. Psychologists have found that social interactions we’re not even aware we’re having can trigger patterns that govern our perceptions and decisions — making us feel happier or sadder, for no reason that has anything to do with ‘who we are’.

Prof. Puett then stated something that I found surprising.

I’ll paraphrase:

‘The world’s biggest companies have figured out these patterns. Last week, I bought a watch online. But this purchase was no accident. Weeks ago, Google picked up that I was thinking about buying a watch. It picked up the style and price bracket of watch I spent most time looking at. And it knows that on Wednesday at 4pm, I finish lectures and have down time, during which I’m more likely to buy something. So it served me an ad, for the type of watch I wanted, at a time when I’m most likely to buy it. And I did.’

His suggestion: smart tech companies have created their products not around our ‘authentic selves’ but around our behavioural patterns, and have worked out how to monetise those patterns.

This is only half true.

My experience of Discovery is that products fail to provide me with anything but linear extensions of my past behaviour - ‘like what’s already been liked’.

In the short term, there’s nothing wrong with this. We have confirmation bias, and do prefer the familiar. To a point. But it’s easy to get wrong.

You’ll see this most innocuously in digital banners. If I spend ten minutes browsing power drills on Amazon, for weeks afterward every digital banner I see features power drills. Even if I already bought the damn drill.

It’s at these times that we experience the ‘uncanny valley’ effect. Algorithms and A.I. can predict our behaviour well enough to be almost right. But they’re also, most often, just a bit wrong. And that reminds us that these entities, these things, do not understand us. Our whims, our curiosity. They can see our behaviour, but don’t understand why we do the things we do.

This is at the heart of the flaw in AI thinking, according to Hubert Dreyfus at Berkeley in California: Machines know that past events, or interests, can dictate decisions in the present through trial and error and the measurement of best outcomes. Whereas humans, most of the time, do things based not on calculation but something closer to a whim, or instinct, which machines can’t fathom.

They can predict our predictable behaviour, but can’t predict our unpredictable behaviour.

So why get philosophical about discovery? Well, because these niggles are at the heart of the digital business model. And they’re symptomatic of a problem with greater gravity.

Fake news, filter bubbles, digital recruitment to ISIS: all driven by a linear process of discovery, that blanks out alternatives in favour of more of what you already viewed.

Myopic ‘discovery’ isn’t just boring us with repetitive content. It’s shaping our view of ourselves and of the world.

So what can we do about it?

Small change, big discovery

We need to design for a type of discovery that is circumstantial, that encourages curiosity, accidents. But immersing users wholesale in opposing viewpoints won’t do it (observe: flamewars on twitter between Liberals and Conservatives). Nor will battering people with content that wilfully clashes with their existing preferences. Prof. Puett offers a solution.

Exposure to minor changes of viewpoint and behaviour — not radical ones — made regularly. This is how they did it in ancient China: regular rituals encouraged them to see the world from a different viewpoint.

These rituals may not revolutionise our world view overnight. But they will help us to see the patterns we have, and so enable us to break them if we choose.

In fact, tech companies — burrowed into every moment of our daily lives, and with knowledge of our previous preferences — are uniquely placed to do this.

And again, innovative businesses are starting to cotton on.

Amazon has found that in disrupting physical bookselling operations, they’ve created another problem. When you can find the exact book you’re looking for, you don’t find the book you weren’t looking for. Call it serendipitous discovery, call it whimsy. Amazon’s web store has eliminated it. And their recommendation engine, whilst robust, isn’t ‘discovery’. Their recommendations eliminate serendipity: what they call discovery is just faster shopping.

So Amazon, king of digital retailers, has built a bookstore.

And it looks a lot like other bookstores. They’ll probably use it to sell a bunch of Prime memberships — but that’ll only work if it’s a good way of discovering your next book. A better, more serendipitous way.

I for one would love to see this ‘little change’ introduced into discovery across the digital business model.

It might make content discovery more like real discovery.

It might return our world of news to something more trustworthy.

It might just change how we see ourselves.