Algorithms don’t predict compatibility

We place some degree of faith in online dating’s “science.” We vote using our dollars on those companies. But should we?

The word “algorithm” has been touted as the savior to relationships and matchmaking, and most of us bought that narrative. We’re getting divorced in greater numbers and getting out of relationships because we’ve got alternatives.

I was one of those guys that believed I’d never try online dating. After getting out of a three-year relationship with my ex last year, I saw her show up on OkCupid as a “95% match.” Those seem like some difficult odds to beat.

As I read Love in the Time of Algorithms–where most of my citations in this post come from–I followed the story of Alexis, who recently graduated from college and moved to New York from Colorado. She’s been experimenting with various online dating sites. She finds her ex on OkCupid as a 90% match and discovers other users with similar stories on an online forum:

“My wife and I split up back in September. She recently told me she joined OkCupid and went on a horrible date. I of course got jealous and joined. Who does it match me up with? Her! Like a 95% match or something. If this was a paid site, I’d be demanding my money back. My female friends are saying it’s a sign we are meant for each other and need to get back together. F*** that.”

I think it’s normal at this stage to question how reliable these algorithms are.

The most surprising thing to me was the lack of innovation in computer or online dating. We’re using same technology–algorithms and accompanying questionnaires–today that we were using when Jeff Tarr created Operation Match at Harvard in 1965, which promised 6 matches in exchange for $3. It used Hollerith punchcards on an IBM 1401.

It also surprises me that online dating companies are intentionally misleading consumers and they know that their science isn’t predictive.

Let’s sell them using scientific claims

If you want to start an online dating site like eHarmony–the site regarded as the most successful–you just need to convince people you’ve figured out all the science.

First you’ll get a picture of a professional on your home page with an official-sounding title and a claim to have “finally cracked the code of human compatibility.” Try something like Relationship Scientist or Sexual Anthropologist. They’ll hold a doctoral degree and write self-help books. They’ll reproduce conditions of relationships, observe interactions, and form conclusions in laboratories. They’ll tell you why certain relationships last or fail. They can even recommend a new book to read while you’re at it. You could hire someone like Dr. Pepper, who has an impressive sounding name and resume:

Dr. Pepper Schwartz, coauthor of Love Test and author of Getaway Guide to the Great Sex Weekend, Everything You Know About Love and Sex Is Wrong, and much, much more, is one such promoter. Dr. Schwartz is a professor of sociology at the University of Washington in Seattle, and in her spare time she poses for PerfectMatch, with whom she helped develop The Duet ® Total Compatibility System. Schwartz says the system is based on the Myers-Briggs Type Indicator, a questionnaire designed to measure how people perceive the world and make decisions. You can read about The Duet® Total Compatibility System in her book, Finding Your Perfect Match: 8 Keys to Finding Lasting Love Through True Compatibility.”

Some web sites go as far as to claim to match you on a cellular level. GenePartner.com claims “Love is no coincidence!” on their home page and will send you a swab kit and find you a genetic soul mate for $249. And the media loves it.

ABC ran a story in awed tones: “Now, hard science is making it easier to find true love. A new matchmaking system uses DNA to help you find your dream date, and it’s redefining what it means to be compatible… Making that first match has always been an inexact science, kissing a few frogs, unavoidable. Until now. With the use of DNA technology, the science of dating has become a whole lot less inexact.”

Academic community fires back

After ABC ran the story on GenePartner.com, the academic community drew the line and called bullshit. Ateam of five Northwestern University professors led by Dr. Eli Finkel wrote a rebuttal. They claimed it’s “virtually impossible to promise to identify potential mates who are uniquely compatible with their users.” The rebuttal was published in Psychological Sciences in the Public Interest.

Here’s where the confusion is caused, and where you’re being misled. For 30 years, we’ve been able to predict with 95% accuracy whether a couple will still be married in fifteen years. Other teams of researchers have reported similar results at that accuracy level in couples already together. This is all thanks to the work of John Gottman, the University of Washington psychologist made famous in Malcolm Gladwell’s 2005 bookBlink.

The marketing used in most dating sites cite this research that predictions could be made, but Finkel claims it’s worthless because Gottman’s marriage prediction (1) assesses a couple on many variables, (2) follows the couple to see if they divorce, and (3) uses a computer to identify the combination of variables that best account for the outcome. However, this is a postdiction, not a prediction.

There are some factors that matter such as socioeconomic status and geography, and those can be used in an algorithm but they don’t reduce the size of a dating pool by much. There are also factors that cannot be known in advance: how a couple experiences and handles stress, especially from situations outside of their control. This includes unemployment, career unhappiness, infertility, and illness.

In Love in the Time of Algorithms, Dan Slater looked at Team Finkel’s critcism on the matchmaking aspect of online dating services:

Sites that claim to offer scientific matching generally adopt one of two perspectives: the “relationship aptitude perspective” or the “compatibility perspective.” The relationship aptitude perspective is more simplistic; it functions on the premise that some people are more likely to succeed at relationships than others. You can spot these people by asking about certain traits, preferences, and aspects of their personal history, such as whether they had positive relationships with childhood caregivers. As the term “aptitude” suggests, this perspective considers relationship ability to be a sort of talent, a quality that, though it can be nurtured and improved over time, remains relatively stable. Team Finkel concedes that the effects of these personality differences on relationships are well supported by research but argues that they leave “most of the variance in adult romantic relationships unexplained.”

The compatibility perspective, on the other hand, assumes not that some people are bound to fail in relationships and others succeed, but rather that there is someone for everyone. The most frequently cited basis for compatibility is similarity–as eHarmony believes. But similarity, Team Finkel says, is difficult to measure: The perception of similarity between two people can have as much to do with relationship satisfaction as with actual similarity. Second, there’s no clear consensus on what kinds of similarity matter. Again, concrete things like religion, wealth, and education have been associated with relationship happiness. But scientific matching sites aren’t claiming to match on these concrete variables. They focus, rather, on psychological variables such as personality, the idea being that two people will succeed romantically if they interact with and view the world in similar ways. In any case, Team Finkel argues, similarity, whether actual or perceived, accounts for only a tiny variance in overall relationship satisfaction. Besides, matching on negative personality traits, such as neuroticism, is unlikely to yield relationship bliss.

In a debate on Valentine’s Day in 2012, Eli Finkel and Sam Yagan both appeared on an NPR affiliate radio station and Finkel criticized OkCupid’s claims:

It is the case that people in same-race marriages are less likely to divorce than people in interracial marriages. But it’s not like these matching algorithm sites are saying, ‘Hey, you’re white. I can introduce you to someone who’s also white.’ No, OkCupid is claiming that they match you to somene who’s similar to you based on psychological dimensions, like personality or values. And the evidence from 80 years of relationship scholarship is just very weak that that actually predicts long-term relationship well-being.

Yagan dodged the question and suggests that Google does the same thing and offers options from a larger pool of information. Google calls them results, and they don’t tell you they’re 90% good or bad results at that. But OkCupid does call the people they recommend as “matches” on their web site, which sugests you’ll get along with them along with a precise percentage.

Dating companies intentionally mislead

What’s surprising to me is eHarmony.com, Match.com, and OkCupid continue to mislead consumers, knowing their algorithms can’t actually predict romantic compatibility. People eat it up.

Former Match.com VP of Product and Strategy Brian Bowman said, “When eHarmony took so much market share in the early 2000s, it was clear that there was absolutely no validity to personality profiling. But eHarmony was doing a huge marketing push. The good white-haired doctor was big presence on TV. So we had to offer what he was offering.”

Amarnath Thombre, the chief of algorithms at Match.com says that only a few parameters actually matter when suggesting matches to people. There have been plenty of matches where 100% similarity resulted in breakups or divorce. Only a few factors actually matter: age, height, political affiliation, smoking vs non-smoking, wants kids vs doesn’t want kids. And even for some of these, users are willing to compromise or negotiate on, even if they don’t realize it. Thombre says, “When you watch people’s browsing habits–their actual behavior on the site–you see them go way outside of what they say they want. The mind makes a complex judgment that can’t really be defined. The idea of dissonance comes into focus.”

Dissonance is when someone says they want one thing, but they actually want something else. The unromantic truth is that we don’t always know exactly what we want, and we’re willing to bend those rules, exchanging one attribute for another. Physical appearance and personality traits aren’t going to hold a couple together through bad times, and they’re only shallow similarities that will at best establish baseline chemistry. Sometimes we know that we want kids or we don’t, and we won’t consider that difference in someone else–although that belief could also change in the future.

Buyer beware

Only 2.4% of the Match profiles are active and can respond to messages. The concealment of your identity means that dating sites can have fake profiles or profiles without an update on the relationship status. “In all cases,” says former Match.com VP of Product and Strategy, “the concealment of your true identity becomes the site’s main asset.” Bowman got married, but ten years later, his Match.com profile is still live on the site. Those who looked at it can only see that he hasn’t been active on Match in the past three weeks.

Match has 75 million registered accounts, but only 1.78 million of them are paying users according to the 2011 Financial Times article Inside Match. Only paid users can reply to messages. This was originally pointed out in the OkTrends blog post Why You Should Never Pay For Online Dating, which was taken down on the day they were acquired by Match.com and its parent company IAC as Dan Slater points out:

Blatt, the man Yagan was trying to convince to pull out his checkbook, unloaded with a stream of sentiment that you wouldn’t exactly call collegial. Outside the conference room, OkCupid’s employees suddenly took a great interest in their keyboards. Who was this person keelhauling their fearless leader? It was their new boss, Barry Diller’s right-hand man, and it was just business, nothing personal. On the day the deal was announced, Yagan removed the piece from OkCupid’s site. When asked about the capitulation, he back treaded. “I chose to take that down,” he said. “Match didn’t ask.” The article’s claims–that Match profits by duping its customers–were pieced together from public information, he said.

What does the future of dating look like?

“Sort of like a social-networking version of Moore’s law, the rate of sharing will double every year. In 10 years from now, people will be sharing about 1,000 times as many things as they do today.” - Mark Zuckerberg

Starting a dating startup today is very risky. Some investors are skittish when it comes to dating startups because they call it a “user acquisition game,” as if the customers are pieces of meat that come with price tags and churn rates. Nearly half of all the people in my inbox on OkCupid are now inactive and I’ve only been there for 7 months. If you want to start a dating site today, you better have strong conviction to hang your ambitions on because the most likely outcome is failure. Just a few weeks ago, OkCupid Labs shut down one of its own dating products as Tallygram joined the deadpool.

The technology hasn’t moved much since 1959 when two Stanford engineering students created Happy Families Planning Services on the IBM 650. To illustrate this, when the 2011 iDate Conference announced OkCupid as the winner for the Best Up and Coming Dating Site, OkCupid CEO Sam Yagan mumbled to Dan Slater on his way to the podium: “Seven years in the business and we’re still ‘up and coming.’”

A misguided belief and faith in “algorithms that predict romantic compatibility” is holding us back. We need to let that go. Algorithms play a small role in our search for love, but the machines aren’t fortune tellers. Meeting people online is now the #2 most common way people meet, with the #1 being through friends. Maybe online usage needs to more closely resemble offline usage.

Online dating must prevent vanity usage from women and punish window-shoppers. Alexis, covered in Love in the Time of Algorithms, describes this behavior and I assume she’s referring to dating sites:

With wit, freckles, a fun sense of humor, and a body toned by frequent yoga, Alexis was confident in her appearance and only a little self-conscious about her jowly cheeks. Men were attracted to her, she knew. She just needed to learn how to avoid the bad ones. But sometimes Alexis got lonely. So she spent time on social-networking sites, where any decent-looking woman could get the attention of at least a dozen guys. “The attention,” she said, “revives me.”

Identity–not anonymity–will play an increasingly important role. The only place identity works against you is when you’re cheating, but it helps you everywhere else. The home page of AshleyMadison.com touts “18 million anonymous users.” 33% of all people eHarmony rejects are already in a marriage, which is good that they ask, but how do they verify that the user is being truthful if they say they’re single?

What does the future of dating look like? We’re not entirely sure, but we’ve come a long way politically and culturally since the 60s. We struck down the ban on contraceptives in 1965 with Griswold v. Connecticut, race-based legal restrictions on marriage were lifted in 1967 with Loving v. Virginia, and we gave women the right to decide if they wanted an abortion in Roe v. Wade in 1973.

The solution will most likely combine identity, trust, and efficiency, because that’s what we use in the offline world. People are usually with friends when they meet new people.

So far, the technology hasn’t replicated that experience successfully.