Data in Dating: From eHarmony to Tinder & further
Big Data is something we are using more and more. It comes with great ideas and solutions but also with uncertainty. The idea behind analyzing data is already quite old. Ian Ayres discusses it already in 2007 in his book Super Crunchers. He discusses how quantitative analysis can be used, in a creative way, to give more insights into all different aspects of life. As data science is way more used nowadays, we will reflect upon a certain case, about eHarmony, in the Super Crunchers book. We will reflect upon the case and compare it critically with recent scientific literature. First, we will give a brief summary of the case, then we will go over the recent literature and critically evaluate the cases.
A summary of the eHarmony case
Ayres (2017) discusses in Super Crunchers the case of eHarmony, a modern — this was written in 2007 — dating service that uses hidden variables to find compatible partners. The founder of eHarmony Neil Clark Warren based its business on his late 1990's study of more than 5000 married couples. He then patented a predictive model — a regression — based on twenty-nine variables allegedly best denoting the perfect relationship: variables on emotional temperament, social style, cognitive mode and relationship skills.
Surprising to Ayres is how eHarmony dared to select the relevant predictors in the form of “hidden variables”, i.e. factors that their own customers were not aware of. As such, it may just happen that their algorithm will match you with someone you would have never imagined you liked. This effect is exacerbated also by the large volume of input data — the customers would have to fill out a 436-question form upon subscription- and the very secret nature of their predictive model.
The competitors of eHarmony use a different paradigm in finding couples. Instead of matching you with the most similar partner, Perfectmatch.com and True.com use compatibility-based methods. Perfectmatch.com is bases its method on the Myers-Briggs personality test, which consists of labeling each individual into one of sixteen personality types. The relatively long record of this test and its growing popularity have also allowed for historical records that state which personality type works best with which. True.com calculates instead the likelihood you will get along with someone else, their model being based on 99 variables.
So, which one is better? Similarity- or compatibility-based matching? Ayres considers that data should be able to adjudicate whether similar or compatible people make the best couples. Well, the truth is no one knows. The algorithms and the data are essential to this industry and therefore kept secret, so it is hard to say. Nevertheless, e-Harmony claims that their married couples are “significantly happier” than any couple that would have met otherwise. No back-up has been found for their claim (except for Clark & Snow (2014) written by eHarmony and kept private). This fact does not stop Ayres from being pleased that, although the best paradigm is unknown, dating services are competing on finding out whether their algorithm got it right as well (“validation” in modern jargon), beyond just on their matching algorithm.
Indeed, for eHarmony, the raw data and the algorithm output does not exclusively determine the final coupling decision. Apparently, even though the extensive questionnaire indicates a homosexual partner preference, e-Harmony refuses to match same-sex couples. Unlike its competitors, e-Harmony insists on matching couples based on similarity, and yet ironically when it comes to gender, opposites attract. Consequently, eHarmony is willing to facilitate only some types of legal marriage regardless of what the algorithm says. In terms of race however, the platform does allow customers to indicate their preference. Since the algorithm is not public, it is possible that eHarmony “puts a normative finger on the scale” to favor certain clients.
The main question
The main question we take up from the case described in Super Crunchers is whether algorithms and statistics can benefit a successful couple, as opposed to the alternative — say meeting someone at work or at a party.
Background
Before the advent of technology as we know it today, or even as Ayres knew it a decade ago when writing about this case, people used to be able to find a life partner in more traditional ways. Dating without an immediate goal of marriage only began in the 20th century, and with it, love and romance started to follow.
With print media being a popular source of information for the ordinary citizen, singles looking for dates could promote themselves using ads in local newspapers.
Only in the second half of the 20th century, Harvard students Jeff Tarr and Vaughan Morrill used a questionnaire and an IBM 1401 to match students based on their similarities, a social experiment that was named “Operation Match” and that would require 3€ per participant to take part in[1]
“We’re not trying to take the love out of love. We’re just trying to make it more efficient.” — Jeff Tarr, co-creator of Operation Match in 1966
Operation Match is estimated to have been used for over 1 million users during the 1960’s. Only a few decades later, with the rise of the internet did online dating really shape up. Kiss.com becomes the first modern dating website in 1994, followed by match.com in 1995 [1] The traditional dating website usually includes an entry questionnaire, a mathematical matching algorithm and requires customers to pay a fee in order to make use of the service.
eHarmony was in fact the first such dating website that includes an algorithm at the core of matching, and the topic of our case file. The case must have been mentioned as a super crunching example as people were just starting to wonder whether matching algorithms really constitute a legitimate substitute for say, instant chemistry at a party.
Scientific Reference set and Descriptions
Method and System for Identifying People who are Likely to Have a Successful Relationship
As described above eHarmony has turned the dating business into a super crunching business, by using data and algorithms to find your perfect match online. The following summary of a patent filed by eHarmony in 2004 sheds some more light on how they match using the algorithm scientifically. eHarmony has been in business since 2000[2], and filed for a patent describing the method and system for identifying people who are likely to have a successful relationship[3] in that same year. The patent filed in 2004 is a continuation of this patent, i.e. an updated version.
The patent describes the whole platform of eHarmony, from their matching procedure to the different forms of communication between matched people on their platform. We will summarize the matching procedure only, as that is the case the Super Crunchers book is concerned with.
Based on a survey that asks questions that can be numerically answered, eHarmony makes a ‘empirical database’ containing the answers of every participant. To build the matching service eHarmony asks ‘members of couples or previously existing couples’ to fill out the survey as well, as to generate a training dataset for their matching algorithm. From this empirical database a correlation matrix is constructed, to show the degree of correlation between the question answers, i.e. variables. Correlated variables are combined into factors, the authors say by possibly using principle component analysis. These scores of every individual on these factors together with his or her ‘individual satisfaction index’ (ISI), how satisfied one person is in his or her relationship, are again stored in a database. The factors and ISI are used to estimate a new joiner’s satisfaction index, this is done using multiple linear regression models. The models give an estimation of the individual satisfaction index based on the survey results.
Next to the ISI, eHarmony computes a ‘couple satisfaction index’ (CSI), this is done using the differences between a couple’s variables, i.e. survey responses. The CSI is estimated using multiple linear regression and is the satisfaction of a given person in a relationship with a specific person. So, where the ISI is a general relationship satisfaction level, the CSI is specific to one relationship. The ISI is then used to cluster people into certain groups with similar ISI levels, from these clusters eHarmony selects the people that have a high CSI level with this person as a ‘match’.
After the matching procedure eHarmony intends to keep in contact with the (potential) couple, possibly sending out surveys to retrieve a measure of relationship satisfaction and thereby of outcome satisfaction, this would in turn be used to update the matching algorithm with new feedback and improve future results, this means the matching service would “learn” by taking into account the result of previous matches.
[2] See https://www.eharmony.com/history-of-online-dating/
[3] See Patent number US6735568
Similarity, Convergence, and Relationship Satisfaction in Dating and Married Couples
eHarmony has a research department, eHarmony Labs. Over the years they have published papers claiming the importance of personality similarity in relationship satisfaction. This is a summary of one of the papers published by eHarmony labs in the Journal of Personality and Social Psychology in 2007 by G. Gonzaga, Belinda Campos & Thomas Bradbury.
Their main results indicate that “similarity and convergence in personality may benefit relationships”. Relationships benefit from similarity and convergence in the couple’s personality by “promoting similarity and convergence in partners’ shared emotional experiences”. Research supporting the statement that partner similarity is beneficial for the relationship helps eHarmony substantiate their claims as their matching system is based on similarity of personality in the first place as mentioned by Ayres in Super Crunchers.
This study uses a sample of college-age couples and newlywed married couples to test their hypothesis that “similarity in partners’ personalities, over and above partners’ personalities considered independently, increases the likelihood that they will have similar emotional experiences” which should promote more “fulfilling relationships”. They test this in three steps namely, they test the claim that partners are similar in personality, then they test the hypothesis that personality and emotional similarity are related to each other and then test whether this is related to relationship quality. Relationship quality was measured using three scales, all based on surveys given to the participants.
Gonzaga and colleagues find support for all three of their hypotheses, meaning that partners were “similar in their personalities and emotions, that personality and emotion similarity were significantly and positively correlated to each other, and that personality and emotion similarity positively correlated with relationship quality”. They also find that these effects are context independent. Next to this, in the two studies they conducted, evidence was found for the fact that similarity in emotion mediated the relationship between personality similarity and relationship satisfaction.
The paper also finds evidence for personality and emotional convergence between partners, meaning they become more alike in those fields over time. Their analysis of the data suggests that “converging and diverging have significant ramifications for the relationship; converging bodes well and diverging bodes poorly”. All in all, this study poses evidence that personality similarity and convergence promote relationship satisfaction, this effect is mediated, amongst others, by emotion similarity and convergence.
As said, this is valuable scientific evidence for eHarmony as their matching system is based on similarity, only users with similar outcomes in the individual satisfaction index are a possible match for each other.
Recommender System for Online Dating Service
In their work, Brozovsky and Petricek (2007) present a recommender system for matchmaking on online dating sites based on collaborative filtering. The recommender algorithm is quantitatively compared to two commonly used global algorithms for online matchmaking on dating sites. Collaborative filtering methods significantly outperform global algorithms that are employed by dating sites. Furthermore, a user experiment was carried out to understand how user perceive different algorithm options.
Recommender systems have been vastly discussed in literature, however, have found little application in online matchmaking algorithms. The authors state that many online dating web sites have utilized traditional offline matchmaking approaches by agencies, such as questionnaires. While some online dating services, for instance date.com, match.com or Perfectmatch.com, have found success in online matchmaking, their algorithms are inherently simple. As an example, an algorithm may preselect random profiles on conditions, like men of certain age, and users can rate their presented profiles. Commonly, algorithms of aforementioned web sites are global mean algorithms.
Brozovsky and Petricek compare four algorithms, namely a random algorithm, mean algorithm (also item average algorithm or POP algorithm), and two collaborative filtering methods user-user algorithm and item-item algorithm. The authors test the algorithms on the Libimseti dataset originating from a Czech online dating website (http://libimseti.cz). The dataset consists of 194,439 users and 11,767,448 ratings of profiles. The dataset is noted to be sparser than widely popular dataset from Movielens and Jester with a sparsity of 0.03%. Nonetheless, it is larger in the amount of entries. To benchmark the algorithms three cross-validations measures are employed. Each validation measure uses negative mean square error (NMAE) as a metric. The cross-validations are AllButOne validation, GivenRandomX validation, and production validation. For the AllButOne validation results user-user collaborative filtering algorithm performed the best with mean algorithm performing notably on similar level “due to strong component[s]” in user preference. In the GivenRandomX validation results user-user algorithm achieves again the lowest NMAE. Validation in a production setting did not provide any surprising results. The collaborative filtering algorithms, specifically user-user, outperformed other competitors.
Brozovsky & Petricek conducted a user experiment to investigate how users perceived the algorithms. Random, mean, and user-user algorithm were tested. Two lists of recommendations were shown to users originating from two algorithms. Between all algorithms, user-user outperformed other algorithms. The mean algorithm, however, performed surprisingly well. The random algorithm performed expectedly poorly.
Brozovsky & Petricek showed in their work that collaborative filtering algorithms, like user-user or item-item are a favorable option for online matchmaking. In general, these algorithms outperform commonly used mean algorithm employed by dating websites and should be considered. Another indication to use collaborative filtering methods is how users perceived the presented algorithms. The acceptance of collaborative filtering was the highest for user-user.
Tinder and the new online dating era
Love me Tinder: Untangling emerging adults’ motivations for using the dating application Tinder
Tumter, Vandenbosch and Ligtenberg shed some light upon the question why emerging adult use tinder. They use a survey among Dutch emerging adults to investigate the different motivations to use Tinder.
Tinder is a relative new kind of dating app and is currently one of the most favorites. Tinder has gotten itself a reputation and is often called the sex-app. The app is one of the first dating apps that is specifically created as a smartphone app, and not just as an extension of an already existing dating site.
Tinder uses personal information of an individual’s Facebook account to create matches. This is information like age, friends, interests, gender etc. The users of the app also must give information about what they are looking for in terms of gender, vicinity and age. The app also uses the GPS function to find matches in close range. Users of the app can base their decision about a potential partner based upon the profile picture and their interests.
Previous research has stated that users of dating websites often have a diverse set of motivations. However, it is still unclear what kind of reasons emerging adults have for apps like Tinder. Other literature shows that motivations for dating websites can be shared across platforms, while other motivations can be unique to certain platforms.
There are different kind of motivations to use Tinder. The 3 main categories are, physical gratification, social gratification and psychosocial gratification. These three categories fall under the Uses and Gratifications theory and can explain why adolescents are using Tinder. However, the main goal of this study is to identify specific motivations of emerging adults who use Tinder.
The survey was distributed among the network of students who used their social media account to distribute it. A total of 266 people participated in the study. The survey was designed to gain insights into the different categories of the Uses and Gratification theory.
The study found that emerging adults often use tinder for excitement and because of the novelty of the app. Tinder is also more often used to establish steady relationships than to find a sexual encounter. The study also found that overall gender and age can account for differences among the motivations. Therefore, motivations to use the app can change when the user gets older. The findings of the study suggest that the outcomes of new technologies like tinder will be highly related to the goals of the users. This study has been the first one that shows that Tinder should not only be seen as a hookup app, but as a tool that is able to satisfy different kind of needs among emerging adults.
Screened Intimacies: Tinder and the Swipe Logic
David and Cambre (2016) scrutinize the swipe feature of the dating application Tinder. In two parts David and Cambre describe, firstly, sociotechnical dynamics when users navigate the user interface of Tinder and the influence of the swipe feature. Secondly, the authors investigate how the swipe disrupts intimacy of online dating.
Tinder is a location-based real-time dating application. A major user interface element of Tinder is its swipe feature. Users see pictures of other individuals and with a simple swipe can either like or dislike the other person. If both users like each other they have a “match” and can enter a private chat.
In their article David and Cambre discuss how the swipe feature of Tinder leads to screened relations of intimacy. The authors chose this term because of the way users interact with the swipe has implications to their behavior. The feature allows for mediatization and depersonalization due to swipe pace. A user is only presented with images such that a deeper understanding of the other does not occur. It is noted that an algorithm dictates the swipe logic such that users are forced to have profile that can catch attention various ways, such as being humorous or witty. The lack of information promotes transcendence over being honest and faithful to oneself to be successful at the “game”.
Nevertheless, it remains a superficial activity, which due the swipe pace may even be used to waste time. The authors attribute such behavior to the immersive mobile culture in which the touching of a mobile device’s screen has become an essential cultural habit for users. Reviews of users about Tinder as dating app reaffirm the time-waste behavior. Furthermore, it is less so considered as a dating app but rather a matter to find one-night stands. Users call Tinder “beauty contest plus messaging” or “McDonald's for sex”. Other articles also mention terms like this, for example in Finkel, E. J. (2012) & Sprechers and Sumter, Vandenbosch & Ligtenberg (2017) where they call Tinder a hookup app. In contrast to the case file, Tinder displays a very different view towards dating than introduced by eHarmony. eHarmony’s goal is it to match partners based on how complementary and how well they fit together, and eventually lead couples to “successful” marriages. Founded in 2000, eHarmony made use of regression techniques, which may be considered outdated nearly two decades later. While Tinder’s algorithm is not public known, it may very likely be much more advanced than eHarmony’s regression. Based on being a location-based real-time dating application factors such as complementary features of couples may be insignificant for Tinder. Features, such as swipes, likes or dislikes, or possibly advanced image analysis of user’s profiles may be found in Tinder’s algorithm. Advancement in data science methodology make eHarmony seem very outdated from a technical point of view. Furthermore, their take on matching couples for longevity and eventual marriage seems archaic in today’s fast-paced society.
Follow-up
Since 2007, follow-up on eHarmony’s revolutionary matching algorithm has taken several directions. On one hand the scientific community is now realizing there might not be any concrete evidence behind traditional matching algorithms, so for this reason but also simply due to new technologies available, those who want to make a dating business are reprofiling to mobile apps, real-life dating and various creative features. On the other hand, in the media and pop culture an increasing number of stories, movies and opinions have emerged on the topic of online dating and are growing more popular with especially young, digitally literate audiences.
Algorithms behind dating services
“We, as a scientific community, do not believe that these algorithms work.They are a joke, and there is no relationship scientist that takes them seriously as relationship science” — Eli J. Finkel, an associate professor of social psychology at Northwestern University. ”
Firstly, one line of scientific follow-up is that of debunking the algorithms behind dating websites such as eHarmony.
“We, as a scientific community, do not believe that these algorithms work,” said Eli J. Finkel, an associate professor of social psychology at Northwestern University. To him, dating sites like eHarmony and Match.com are more like modern snake oil — “They are a joke, and there is no relationship scientist that takes them seriously as relationship science” [4]
Mr. Finkel spent more than a year with a group of researchers trying to find backing to the claim made by computer dating services and after investigating more than 80 years’ worth of scientific research on dating and attraction, he was unable to find concrete evidence in favor of websites such as eHarmony [4]
First one to use an algorithm
Since e Harmony was the first dating website that based their matches upon an algorithm they did have some scientific impact (Finkel, 2012). After eHarmony other dating websites like Perfectmatch.com and FindyourFaceMate.com followed their lead (Finkel, 2012). The users of the sites then thought that they had found a superior way to find the perfect partner. This led to different studies that investigated whether online dating is as good as offline dating. It was found that online dating is good for singles but not better than offline dating and could in certain circumstances even be worse than offline dating. This shows that the eHarmony case did have some influence on other dating services and the way they evolved.
Most of online dating websites claim all kinds of things. For example, eHarmony claims that their algorithm can find a uniquely compatible mate for every single and they state that their married couples are more compatible than other married couples. The problem with these kinds of claims is that they are not supported by credible evidence. There actually has been some criticism about their claims, for example in 2018 the advertising standards authority stated that an ad that eHarmony was using was misleading (Advertising Standard Authority, 2018). When the ASA ask for support for their claims made in the advertisement eHarmony was not able to give any evidence for their claims.
eHarmony was a dating website who dared to do something different, this could have made the road easier for non-traditional dating tools like Tinder. Tinder, however, still has a reputation as a hookup-app. Although, society has started to accept the use of Tinder to find a partner more and more. EHarmony is in a way also very controversial, it for example does not match people of the same sex. This is nowadays way more accepted and therefore new kinds of dating tools do match people on the same sex. And nowadays eHarmony does match people of the same sex through their sister company called CompatiblePartners.
Modern features
Secondly, several more online services have taken up, mainly Tinder but also Bumble, Hinge and recently Facebook Dating. Each of these apps have to some extent scientific reasoning behind them, mostly incorporated in the user experience. The most significant difference from eHarmony is the use of GPS data to track potential partners nearby, known as location-based real-time dating. More new additions include the infamous swiping feature in Tinder, or answering a set of questions in your profile in Hinge, or even formalizing a sense of humour — endeavor incorporated by Lalife[5]or Huamor[6] by allowing users to rate several funny videos in setting up their profile.
[5] See https://laflife.com/
[6] See http://www.huamor.com/
Tinder versus traditional dating websites
The dating app Tinder is found to be the one of the most popular dating tools. It has been one of the most downloaded in the category lifestyle app in America. The social psychologist that writes the paper Purvis (2017) has been analyzing hundreds of surveys about user experience for Tinder.
Most more traditional dating websites like eHarmony or Match.com try to connect people based upon similarity. Tinder however uses geolocation, and a swiping system. Tinder matches are made with sparse criteria namely location, looks and availability. Psychologically, Tinder is constructed in a way to encourage a rapid swiping behavior. The reward, a match, can happen with every swipe but you don’t know which swipe. Tinder uses a variable ratio reward schedule. This means that true potential match will be randomly spread among the other matches. This is the same kind of reward schedule that is used is for example slot machine and video games. Therefore, Tinder may even be seen as a game. It is also found that the serial swiping behavior seen in Tinder users can develop and feel like an addiction.
While Tinder is often used for short term sexual relations a study shows that the most common motivation for using Tinder is actually to find love. However, Tinder users also experience more frustration about their romantic encounter than users of traditional dating websites.
The main difference here between Tinder and other more traditional dating websites is that their swiping mechanism can be regarded as a game and even can be addictive.
Clearly, data about Tinder users can give lots of insight in human dating behavior. Some people have been saying that Tinder has caused a “dating apocalypse” (Pervis, 2017). However, this paper makes it clear that there is no clear difference in the sexual behavior of people using Tinder and users of traditional dating websites. The main difference here between Tinder and other more traditional dating websites is that their swiping mechanism can be regarded as a game and even can be addictive.
Facebook Dating: Back to basics?
In September 2019 Facebook has released in the United States their own version of a dating app, Facebook Dating. The rationale behind this new venture is that, since you have been feeding data, pictures, updates for years being a Facebook user, the dating app would be able to recommend matches that are presumably more authentic than the standard swiping apps. [7] Moreover, although the app will not match you with any of your Facebook Friends out of obvious reasons, a new feature allows you to add 9 of your friends or Instagram followers on a Secret Crush list. Does this mean we are back to dating people we know in real life? Back to basics? It would indeed add a twist to the contemporaneous online dating arena. However, a journalist reviewing this feature is determined she will never use Secret Crush as “years of app-dating broke my brain and now I’m only capable of being attracted to strangers on the internet”. [8]
[7] See https://newsroom.fb.com/news/2019/09/facebook-dating/
[8] See https://www.insidehook.com/article/internet/review-you-definitely-dont-need-facebook-dating
Where are we headed?
Thirdly, the non-scientific media is becoming more and more active and outrageous and is mostly instantiated in the form of movies and series such as Black Mirror’s futuristic rendition of mandatory automatic partner matching experience.
Netflix’s Black Mirror show is one of the most popular technological dystopias. It has been graded with an 8.9 out of 10 on IMDB — same as Pulp Fiction, for reference. Each episode illustrates an extreme scenario in which technology has changed and challenged our societal norms to an absurd degree. One depicts contact lenses that record everything, after which the video of happiest memories can be watched on a screen or even replayed in your head. Another includes a doll that stores the personality of pop singer Miley Cyrus and is in fact a mini version of her, with her jokes and charm and voice included — this doll then helps save the real Miley from a coma. Another shows a virtual reality video game in which cheating on your spouse is possible, even as your game character has the opposite sex — touching upon sexuality issues as well.
One episode — “Hang the DJ” — concerns a futuristic version of dating. It is at the moment of writing the second-highest rated episode in the shows’ entire 5 seasons. In short, Amy and Frank live in a walled-off society where people are required to be matched into romantic relationships. All relationships come with an expiration date, that is only revealed if both partners chose to do so. A digital Coach (think, Siri or Alexa) collects the data from failed relationships and helps people find the ultimate compatible other. Amy and Frank met for just 12 hours before being paired off with others. After a few brief encounters, they realize they are in love and they rebel against Coach and the System.
What is new about the plot of the episode is the mandatory pairing of couples, made possible by the ubiquitous data collection of the omnipresent System. Even bad relationships — that may take either 1 week or 1 year — reveal useful information, i.e. ‘training data’.
In another Netflix series, one-season French language Osmosis, a firm specialized in neuroscience promises a set of beta-testers that with a brain implant, they will find their true love. After finding said partner, should they choose to adopt the implant as well, the couple would be able to enter a semi-psychedelic heaven-like state of togetherness as soon as they touch a sensor under their wrist. Ultimately problems arise, such as couples breaking up and more dramatic plot twists.
Still, it is not hard to see how in a few decades as neuroscience becomes more advanced, there might be some people trying to bring this to life.
Advances in dating technologies may very likely be a future endeavor of many businesses trying to keep up with new developments, be it in terms of better product design and user experience, or in more far-fetched ways such as those from movies, now only fictional.
Discussion and Critique
Myers-Briggs debunked
“We don’t claim to evaluate you perfectly, but we do claim to find someone who claims to fulfill your claimed requirements, exactly.” — OkCupid
And so, it seems OkCupid makes a great point — it admits that they can’t evaluate someone perfectly. Well, if you are not perfectly evaluated, then your potential perfect partner candidate is also not perfectly evaluated. Yet they claim that somehow these imperfections are evened out when the couple gets together? Unlikely.
When it comes to evaluating an individual accurately, it seems that there is more than meets the eye. As stated by Ayres in Super Crunchers, a competitor of eHarmony, True.com, uses the Myers-Briggs personality test in order to categorize its customers in the right box — one of 16 typologies, based on binary classifications of four traits — to use later in their algorithm. In the meantime, however, Myers-Briggs has been effectively debunked.
Originally, the reasoning behind the Myers-Briggs test stems from Jungian psychology, theories that in the beginning of the 20th century were more in the domain of “hunches” and not empirically tested[9]. Moreover, the namesakes Katherine Briggs and her daughter Isabel Briggs Myers never had a formal training in psychology[10]. Several analyses have shown that the test does not accurately predict career choice, and more than 50 percent of users have different results on a second take, even as the second test is done as early as five weeks later[11] . The test has also been widely discredited by psychologists. CPP, the company that publishes the test has three psychologists on their board, none of whom have used the test in their publications. “It would be questioned by my academic colleagues,” Carl Thoresen, a Stanford psychologist and CPP board member, admitted to the Washington Post in 2012[12]
The Myers-Briggs classification into “types” nevertheless remains one of the most popular psychology tests today. Allegedly 89 out of Fortune 100 companies, along with 200 US federal agencies including the CIA, State Department and the military administer Myers-Briggs test to better train their employees[13]. A combination of excellent marketing from CPP and the feel-good, vague enough descriptions of the types (known as the Forer effect, also occurring in astrology and fortune telling)[13] explains its continuing popularity in business.
Indeed, accurately evaluating an individual is a hard business, especially if one chooses the Myers-Briggs like True.com. And that’s just one part of the story. Even under the assumption you can perfectly describe an individual in terms of personality, then assessing compatibility is a different, arguably even more complex story, as more variables come into play. The scientific backing of compatibility should therefore be even stronger.
See [9] https://www.vox.com/2014/7/15/5881947/myers-briggs-personality-test-meaningless
See [10] https://www.capt.org/mbti-assessment/isabel-myers.htm?bhcp=1
See [11] https://journals.sagepub.com/doi/abs/10.1177/014920639602200103
See [12] https://www.washingtonpost.com/national/on-leadership/myers-briggs-does-it-pay-to-know-your-type/2012/12/14/eaed51ae-3fcc-11e2-bca3-aadc9b7e29c5_story.html
See [13] https://www.vox.com/2014/7/15/5881947/myers-briggs-personality-test-meaningless
Super Crunchers Critique
Ayres has simplified the case to make it understandable for the general public reading this book, most of them likely have limited “Super Crunching” or data science experience. In general, the simplification that Ayres made is a fair one. Not mentioning the exact procedure as described in part 4.1 of this article, the summary of the eHarmony patent, is a good choice to prevent this book extending over 1000 pages.
Interesting is that he fails to mention the construction of factors out of all the variables resulting from the survey filled out by candidates for the program. In the patent eHarmony describes the method of principal component analysis (PCA), it seems quite important for the reader to understand that the matches by eHarmony are not made just using “29 emotional, social, and cognitive attributes” but are made by compiling way more variables, i.e. answers to survey questions, into 29 (very abstract) emotional, social and cognitive factors.
Also interesting is the fact that Ayres mentioned predicting couple compatibility. We think that he is missing an important point here, people do not get matched on compatibility, they get matched on “relationship satisfaction”, how satisfied they are in any relationship. It is then assumed by eHarmony, and in this case Ayres, that this means that people are compatible.
This brings us to a more significant problem, as Houran, Lange & Rentfrow (2004) claim, eHarmony’s scientific basis is “not referenced in detail and a copy of the full analyses and results are neither posted for customers nor otherwise offered to the public for evaluation”. Later Houran, Lange & Rentfrow (2004) expand their claims by stating that the paper that is also mentioned in Super Crunchers, Carter & Snow (2004), is questionable in its experimental design and used statistical techniques [14]. An example of this as mentioned in Houran, Lange & Rentfrow (2004), the results of the study by Carter & Snow (2004) indicated that eHarmony does not match on similarity as the data suggests eHarmony couples are more dissimilar than the control group. Ayres mentions that this study has its deficiencies but to us it goes over too quickly and waves it away as good intentions while it might be a way to deceive eHarmony customers.
Maneuvering in this field of research is difficult however, as there are several online matchmakers that publish their own research supporting their own claims. The paper by Houran, Lange & Rentfrow (2004) is based on research conducted by True.com and Houran and Rentfrow both work for True.com. This shows that every claim by these papers should be considered with a skeptical mindset and we believe in general Ian Ayres managed to give the reader a relatively objective and accurate insight into the online matchmaking Super Crunching competition.
[14] We were not able to get access to Snow & Carter (2004), therefore we are not able to do an evaluation ourselves.
Timeline of Ideas in Online Dating
eHarmony was one of the first companies to introduce an algorithm to match compatible partners on an online dating platform. The company paved the way for data crunching in online dating. Many followed their approach of crunching data, such as Match.com or Perfectmatch.com. In the case file the performed crunching was a regression model for matchmaking. Even though e-Harmony did not share the underlying algorithm, the business was trusted by people and became profitable only four years after its inception. Today, the algorithm e-Harmony used may very likely be considered outdated or too simplistic. A regression method is not able to capture many of the available data sources nowadays. Social media accounts or other online activity or behavior are an enormous source of information that may find application in an online match making algorithm nowadays. A prime example of a current, more complex algorithm is Tinder. The dating app Tinder performs advanced image processing of user profiles, analyses user behavior within the app and factors in location in real-time, among others. Even though e-Harmony may seem outdated nowadays, it must be noted that the company’s data crunching brought acceptance of such methods to the public and eventually also enabled new comers in the field, such as Tinder, who perform much deeper and detailed analysis of a user’s data.
Post-Scriptum (Extra)
Matching algorithms have an unexpected side-effect… reverse-engineering!
References
Advertising Standards Authority | Committee of Advertising Practice. (2018, January 3). eHarmony UK Ltd. Retrieved from https://www.asa.org.uk/rulings/eharmony-uk-ltd-a17-392456.html
Ayres, I. (2007). Super Crunchers: Why thinking-by-numbers is the new way to be smart. Bantam Books.
Brozovsky, L., & Petricek, V. (2007). Recommender system for online dating service. arXiv preprint cs/0703042.
Buckwalter, J. G., Carter, S. R., Forgatch, G. T., Parsons, T. D., & Warren, N. C. (2004). U.S. Patent №6,735,568. Washington, DC: U.S. Patent and Trademark Office.
Carter, S., & Snow, C. (2004). Helping singles enter better marriages using predictive models of marital success. Poster presented at the 16 Annual Convention of the American Psychological Society, Chicago, IL, May 27–30.
David, G., & Cambre, C. (2016). Screened intimacies: Tinder and the swipe logic. Social media+ society, 2(2), 2056305116641976.
Finkel, E. J. & Sprechers, S. (2012, May 8). The Scientific Flaws of Online Dating Sites. Retrieved from https://www.scientificamerican.com/article/scientific-flaws-online-dating-sites/
Gonzaga, G. C., Campos, B., & Bradbury, T. (2007). Similarity, convergence, and relationship satisfaction in dating and married couples. Journal of personality and social psychology, 93(1), 34.
Houran, J., Lange, R., Rentfrow, P. J., & Bruckner, K. H. (2004). Do online matchmaking tests work? An assessment of preliminary evidence for a publicized ‘predictive model of marital success’. North American Journal of Psychology, 6(3).
Purvis, J. (2017, February 14). Why using Tinder is so satisfying. Retrieved from https://www.washingtonpost.com/posteverything/wp/2017/02/14/why-using-tinder-is-so-satisfying/?noredirect=on
Sumter, S. R., Vandenbosch, L., & Ligtenberg, L. (2017). Love me Tinder: Untangling emerging adults’ motivations for using the dating application Tinder. Telematics and Informatics, 34(1), 67–78.
The History of Online Dating (US). (n.d.). Retrieved from https://www.eharmony.com/history-of-online-dating/
