How Facebook predicts your politics: Aesthetics and glossy die cut prints
With the title “Fashion Models and Cyber Warfare”, Cambridge Analytica’s whistleblower Christopher Wiley gave a presentation in November 2018 at BoF voices, a fashion industry gathering at Oxfordshire. In describing the methods Wiley had used to support Trump’s election in 2016, he had to strike a delicate balance between condemning his previous employers yet celebrating the power of the AI methods he wielded could supposedly have. His message to an audience of fashion devotees was particularly flattering: You are not just working in any industry, but one that has changed the course of history.
The company supporting American conservative causes (in addition to elections in India, Kenya and Mexico) infamously built personality profiles of people based on the likes and other details of their Facebook profiles. Though Cambridge Analytica once proudly claimed that it had thousands of data points for every American voter, Wiley emphasised in his talk that the psychological profiling derived its power from a focus on personal style or aesthetic: “When you look at personality traits, music and fashion are the most informative for predicting someone’s personality.” The affinity with particular clothing labels was a signal to be used to identify individuals susceptible to Trump’s brand of populism, in what Wiley called the “weaponisation of fashion brands”.
Perhaps unsurprisingly for someone who had dropped out of a fashion research PhD, and who would soon after the talk be hired to do AI for the Swedish fast fashion label H&M, Wiley was keen to draw out the links between fashion and politics. His argument that fashion has a constitutive function for movements: Maoist and skinheads alike created their own distinctive aesthetics, before they could become groups that people identified with and that had a clear distinctive identity. That is also why the likes of Cambridge Analytica did not just focus on spreading propaganda but created styles. The company used stolen data to design new aesthetics that its target groups were susceptible to. Fashion choices are a means of distinguishing and identifying individual people, but also a modus operandi for exerting influence over them.
Wiley stopped short of saying exactly which fashion choices characterised the potential Trump supporters they targeted, though he makes an example of Wrangler fans scoring low on both openness and conscientiousness.
The aesthetics typical to the European counterparts of Trump’s supporters are quite different, based on the information that Facebook gives out about its own profiling operations. For this blog post, I studied specifically the groups that Facebook had categorised as supporters of various Finnish parties.
One Finnish Facebook page, KuruKukka (Valley Flower), was created by a self-published author living in Ranua, Finnish Lapland, a municipality above the arctic circle with fewer than 4,000 people. The page supplies a steady flow of images, several a day, with consistent visual pattern: floral arrangements and butterflies, in montages with angels and children. The images are superimposed with text, with a message specific to that day: Have a happy Tuesday! I hope your weekend is starting well! The 33 thousand followers of the Kurukukka give a typical image of the page a couple of hundred shares.
Kurukukka is one of the pages on Facebook that, according to the platform, is most strongly associated with the anti-immigration conservative The Finns party. In a time of elections in Finland, millions of euros are spent on political advertising, and measures of party interest are the most typical way in which these advertisements are targeted. The clicks that land on the images arranged by the Lappish artist play a small and largely unknown part in directing the flow of millions of political messages. It is part of the process in which Facebook abstracts socially meaningful categories from ordinary human interaction — interests, hopes, political positions — and leverages them for competitive advantage in its advertising products.
The list that Facebook provides about the party’s supporters affiliations include a wide range of things, from consumer brands to Facebook pages sharing funny pictures. The list of The Finn’s affiliations includes another Facebook page with a similar visual style, “happines” (sic), resembling KuruKukka except for its focus on animals and small children.
If there is anything like an aesthetic particular to supporters of The Finns, it could be this folksy style, a world of innocence where everything glimmers. Thinking of the cultural associations of this style from a Finnish point of view, the visual genre of “kiiltokuvat” (literally “shining pictures”). Such images were in the in the 19th century among the first colour prints and were often die cut along the figures’ outlines. Scrapbooks called “muistokirja” were passed around to friends and family, who were encouraged to attach both the glossy cut out pictures as well little passages of text. The “memory book” was like a social networking technology of its own time, and the “shining picture” its aesthetic. A similar style can still be seen in postcards.
This association of aesthetic and political attitudes was not discovered by a data scientist, but was rather part of the routine functioning of Facebook’s profiling system. Every user profile on the site has a list of interests attached to it, attributed automatically based on a set of factors that are largely unknown, yet likely include in the least one’s Facebook “likes” as well as surfing behaviour. Cambridge Analytica might have been in the limelight, but profiling is of course the bread and butter of the advertisement industry. Especially when it comes to politics, the categorisation of people and selling their attention enables what Turkish sociologist Zeynep Tufekci calls “computational politics”, turning “political communication into an increasingly personalized, private transaction and thus fundamentally reshap[ing] the public sphere”.
Facebook users are encouraged to view the way the platform classifies them, or in the least the accessible parts of their profiles. These individual categorisations can be scrutinised, though with no certainty whether the same biases or omissions apply to other users. So far little attention seems to have been given to the way in which Facebook defines groups and the representations that it creates of collectives of its users. What do the supporters of various political positions look like, according to Facebook? What kind of knowledge flows into this profiling? And what is the power of these representations of groups? Some answers, in the least speculative ones, can be attempted through with the information that Facebook discloses to advertisers.
Comparing supporters of Finnish political parties
To understand how Facebook creates profiles of and predicts people’s political positions, I compared the people that Facebook classifies as having an interest in different Finnish political parties. Since 2019 includes both general and European parliament elections in Finland, this comparison was topical.
Using some freely available and simple tools, I made lists of the most distinctive connections that Facebook reports for supporters of each party. In the embedding below, I display these lists of pages next to each other so that they can be compared across parties. This presentation makes it possible for readers to browse through a large amount of information in an interactive manner, especially if they are reading this on a desktop.
There are 10 parties currently represented in the Finnish parliament, out of which I could represent 7 in this list. Every column represent the affinities of party supporters, with the most distinctive ones listed on top. The parties are ordered from left to right following the classical representation of the political spectrum.
What exactly is the content of these lists? They are derived from a tool that Facebook provides to advertisers called Audience Insight. It allows advertisers to define potential audiences, such as people of a certain age who live in a certain city and who have an interest in a particular topic. Interests include over 200,000 different labels (including most of the world’s political parties) that Facebook attaches to individual profiles.
For every audience, Facebook reveals demographic and economic details. The platform also discloses the Facebook page likes that are most distinctive for a particular audience, i.e. the likes of various brands, organisations or Facebook communities that are much more common among the defined group than among the public more widely. The magnitude of this difference, between a particular group and other Facebook users, is called an Affinity score (also displayed in Figure 1, the list of pages displayed above).
The information about page likes and demographics is intended to help companies create ads that are relevant to the various audiences they are targeting. As such, they reveal something about the characteristics of the groups in question. But it is possible to read the information in another way too: these are the characteristics Facebook uses to profile and categorise people. Even if someone would not have expressed a direct interest in, for instance, a political party, a user profile with similar kinds of affinities would suggest a potential supporter of the party, a ripe target for advertisements or organic recommendations.
Differences in the political causes connected to parties
When comparing party supporters in terms of their Facebook likes, some connections felt accurate and insightful and others quite arbitrary. The first thing one might notice is the similarity between the Green League and The Left Alliance. The former of these is a left-of-centre party with an ecological focus, while the Left Alliance was formed through the joining of SKDL (effectively the Finnish communist party) and other left groups. Both parties are known to have similar supporters: young, predominantly female, university-educated urbanites. The Left Alliance also has strong links with trade unions and industrial workers, but this demographic didn’t seem to be represented in Facebook’s profiling of party supporters, perhaps simply because of less frequent Facebook use.
More than other parties, The Greens and the Left Alliance are defined through affiliations connections to what could be called causes, i.e. on-line campaigns or non-profits working on social issues. Both of them are fans, for instance, of the League of Finnish Feminists and the umbrella organisation for development aid, Fingo.
Systematic differences between these two parties do seem to exist, especially if you take into account the relative intensity of the links. In terms of policy, the two parties differ most clearly in economic policy, and a similar difference could be read in a description of their supporters too. The Greens are particularly strongly connected to causes related to veganism and companies that specialise in vegan products. NGOs and movements related to LGTBQ issues also stand out. The Left Alliance is connected to causes related to corporate power, such as opposition to the TTIP trade agreement, some of which don’t register with the Greens. Also the connection to anti-racist groups seems relatively stronger, including the page Loldiers of Odin that parodies the growing anti-immigration street patrol activities under the name of Soldiers of Odin.
The Finns, the conservative party opposing both immigration and the European Union, are unsurprisingly connected to Suomi Ensin (“Finland First”) page, a channel for ethnonationalist propaganda. A number of other causes doshow up on their list, including a movement advocating for removing restrictions from the sale of alcohol and a page on the topic of protecting welfare benefits. Keskusta, the formerly agrarian Center party, is predictably connected with the farmer’s and entrepreneurs interest group, but also the war veteran’s and reservist associations. The other parties are more connected to causes that could be described as purely charitable, such as Unicef and charities working in the health care or social work.
Links between lifestyles and political parties
Differentiating between political causes associated with the left or the right might not yet be particularly insightful. The algorithmically generated profiles of party supporters, however, also feature heavily page likes that are more about lifestyle than politics. I have displayed another list of pages likes below focusing specifically lifestyle-related pages (such as food, music, consumer products).
In particular supporters of the Finnish Social Democratic Party (SDP) are identified almost exclusively through a pattern of connections to brands. This might in a small part be the result of a glitch: The most important affiliation for SDP is Halonen, a Finnish textile business. Incidentally, this company has a namesake in the first Finnish president, Tarja Halonen, whose legacy will have been a frequent topic of discussion in the party’s Facebook conversations. One can only speculate, but this might have been enough to confuse the platform’s algorithms.
In other respects, Finnish social democrats seem distinctively keen on various family-run clothing businesses and domestic items, such as cleaning products and medicine. With prominent links to publishers, the party of aging female voters appear to be avid readers.
The Finns party attempt to distinguish themselves as the party of motorists, and possibly because they know their supporters. The Finns supporters frequently like the Facebook pages of gas stations, in additions to domestic meat producers and pages dedicated to discussing the Finnish lottery system. Supporters of the pro-business conservative party Kokoomus also seem to have a keen interest in America, as one of the most strongly affiliated pages is a blog that gives travel tips to New York.
Music does not feature heavily in the page links but provides some interesting details. For the agrarian Keskusta, a pop musician dedicated to the Finnish version of the schlager genre (iskelmä) stands out, while for The Finns its the band of aging punks, Klamydia. The woke hiphop artist Paleface is unsurprisingly associated with the left, while the biggest pop music favourites of the baby boomer generation perhaps unexpectedly connect with the conservative Kokoomus. The Centre party has particularly strong links to professional athletes, particularly people doing skiing.
Many pages make the list that really only exist in the world of Facebook and are dedicated to the sharing of digital content. Particularly The Finns are widely connected to sites that are dedicated to sharing of jokes (in my estimation, of a laddish or raunchy variety).
Also the glimmering images of Kurukukka and happines that I have already described feature on the lists. I will return to them in the end of the text, after some thoughts on the nature of this information.
Statistical knowledge and ”the space of life-styles”
Popular discourse about the power of social media platforms and “big data” analysis often describe something like a revolution in the amount of information that is collected. This change in scale is enabled by attending to the traces left by human behaviour on digital systems, as opposed to information that is collected specifically for research, for instance through surveys. Making use of the data that is collected by platforms such as Facebook, however, does not only imply a change in the quantity of information we can go through, but also in the types of knowledge that we can produce — a change that is also partly an impoverishment of our understanding.
One helpful point of reference for charting this change is Pierre Bourdieu, a canonical figure in sociology, in particular on the theme of aesthetics and politics. Bourdieu’s impressive charts of describing the regularities and patterns of taste and consumption in 1960s French society are perhaps something like big data, to put it fancifully, “avant le lettre”.
What I find joyful about Bourdieu’s books are the graphs and visualisations, produced from large-scale survey data. These often map out habits or practices in two dimensions, as in the graph called “the space of social positions” displayed above. The axes are derived from Bourdieu’s overarching theory, describing how access to different forms of capital — economic, cultural, social — give rise to a single structure that shapes numerous domains of human life, not only to our professional activity but equally what we eat, the films we watch, and which make of a French car we choose to buy. Reading the admittedly obscure picture above, one can discern that cycling is typical for those bestowed with cultural capital and no money, and we will encounter country walks, camping or swimming progressively while looking at the less acculturated.
This is a picture of French society, but it also indirectly says something about the type of knowledge the sociologists are producing. The space lays out a picture that is in some sense complete, attempting to describe the nation state of French as a whole, displaying the entire spectrum of social positions within it. The various different classes, the theoretical categories chosen by Bourdieu, all have their particular places within it.
Even a biased picture makes a difference
A direct comparison of Cambridge Analytica to French sociology would of course be inappropriate, yet in some ways the differences in their approach are telling. In the analysis Chris Wiley described in his presentation, the link between aesthetics and politics is made through individual personality. Fashion is an indicator for psychological types, and these psychological types allow political campaigns to customise their message as well as identify the people that are most susceptible to it. There is, however, no attempt to understand voters as parts of some larger and stable social groupings, be they social class, demographic group or some statistical definitions such as the unemployed.
This may be because of the project’s aims — and Zeynep Tufekci does highlight individualised targeting over reasoning of aggregates as the holy grail of computational political campaigns. The difference also has to do with way that social groupings are represented in the source material, the vast troves of latent trace data from digital platform.
In survey work and statistics more widely, the definition of groups and demographic categories is typically the starting point. The ability for surveys to generalise over and describe entire populations based on sampling is one of its central features. In contrast, the interpretation of the traces left by users of social media platforms doesn’t have any single scale of analysis or fixed categories. The analysis can often aggregate information about the identities that people ascribe to themselves, for instance through the creation and joining of shared pages or hashtags.
Even the parties that I have briefly studied in this post are a group that has been algorithmically identified after Facebook users themselves found it significant. Our comparisons could just as well have focused on the hundreds of thousands of other such interests. It is important to keep in mind how these categories have been derived when reading the list of page likes that I describe above. There is significant overlap between the separate groups of party supporters identified by Facebook, and they don’t behave like well-defined, mutually exclusive statistical categories. Even though the parties are displayed from left to right, in some way like Bourdieu’s mappings, it’s not like what we are seeing is a representation where every single part makes up the entirety of a political spectrum.
This fluidity of groupings and identities could of course be representative our era: allegiances to traditional political mediators including parties themselves is on the wane. As Will Davies points out in an essay on the growing power of “big data” at the expense of statistics, “this is a form of aggregation suitable to a more fluid political age, in which not everything can be reliably referred back to some Enlightenment ideal of the nation state as guardian of the public interest.”
Social media platforms reflect the proliferation of new groupings, but they are also actively producing it. For all its fault, the one undeniable upside of social media is how quick and easy it is to discover likeminded people, form some kind of vague identity and share thoughts. The fact that social media platforms facilitate this, however, points to another fundamental problem about the data collected from them. You can’t think of this technological layer as representing social phenomena, when the likes of Facebook are actively shaping them.
Imagine, for instance, the choice of whether or not to declare your affiliations with a particular party, through something as simple as like of its Facebook page. Whether or not people undertake this action will greatly be influenced by whether or not the platform encourages this choice through a recommendation of the page. Even when visible, a person’s choice may well be skewed by the details Facebook chooses to display, such as the current count of likes or whether one’s own acquaintances have shown their support. Social media provides an effective system social information, telling you what other people around you are doing, opening up the possibility of feedback loops and viral phenomena.
The groups of party supporters I have briefly described could hence be the result of a complicated interaction between digital devices and genuine human dispositions, perhaps resulting in its current composition through some complicated process where early group formation and unexpected events have determined who joins later. It may be impossible to pry apart what could be labelled as the “digital bias” of the platform itself from some kind of ground truth about people. In some ways this kind of distinction might not even be desirable: After all, the collective representations that Facebook have a certain reality of their own. It has real-world consequences, shaping the way recommendations, ad money and attention is channelled. Perhaps it works like a self-fulfilling prophesy, amplifying and making real the platform’s caricatures of political groups.
Aesthetics is not just about your designer clothes
When clicking through the pages associated with political identities, I came across the page of a Finnish comic artist Kaikki Meni, with a following of about 20,000 people. This artists’ work was quite strongly associated with the Left Alliance, the opposite end of the political spectrum relative to The Finns that I described in the beginning of this blogpost.
What made me curious that there was a certain inversion of an aesthetic approach as well. While KuruKukka felt like a fairly direct reuse of the imagery of old postcards and glossy cutout prints, Kaikki Meni uses the twee aesthetic for comical effect. The kitschy style sets up an expectation of harmony, which is broken through absurd statements and cursing. A similar usage of Victorian era imagery in other Internet comics, such as Wondermark: a conflict between form and content, and a parody of the world of kitsch.
It might be possible to read this opposition following the theory of Bourdieu, claiming that a privileged position with cultural capital enables a reflexive and playful approach to different aesthetic systems. I would, however, like to make a different point, referring more to the historical role that these types of images (the glossy cutout pictures) have played in previous systems of communication, and the potential continuity that there is in ways of using Facebook.
When people share images from Kurukukka, their activity is in some ways comparable to sending postcards. The aesthetics of the images is the same. The form of communication is similar: the messages on the images celebrate particular occasions or the passing of time. For the people participating in the sharing of these images, including also the conservatives supporters, using Facebook replicates or is in continuity with earlier ways of communicating.
This observation could provide a key to reading the differences between the groups of party supporters described above. It seems to me that what is differences between particular political groups are not so about preferences in one domain of life (for instance differences in, say, what clothing, music or food they like). Rather what distinguishes the groups is what domains are associated with their group profiles to begin with: The profile of one party’s supporter is heavily interested in company brands, while another focuses on sports, and yet another on pages focusing on humour.
Another way of describing this is that there are significant and systematic differences in the way that Facebook is used. For some its a source of entertainment, others want to show support for causes, while for many it is about keeping in touch with their closest friends, and possibly utilising the imagery of pages like KuruKukka in the process. There is a group of people for whom the medium remains like an extension of older mediums and ways of communicating, while there is also playful experimentation and pushing the boundaries of what can be done on the platform.
The signal that Facebook picks up most clearly in its automated profiling may be related to these patterns of media usage, instead of tracing for instance people’s consumer choices or support for social causes. Facebook is not only a medium that extracts data on people’s preferences — what the system is measuring is what kind of medium Facebook is for people.
When Chris Wiley talked about the potential for fashion and aesthetics for the profiling of people, he was not only talking about the dangers of artificial intelligence but also making an argument along the lines of consumerist ideology with a long history, claiming that the our consumer preferences and behaviour in the market reveal something about people that is fundamental. This in itself may not be Wiley’s greatest crime, but its curious to see that the Facebook’s profiling algorithms themselves have picked up on a wider sense of aesthetics, which extends to the more mundane world of meme pages and montage postcards.
Aleksi Knuutila is a digital anthropologist exploring new methods for scrutinising political culture on digital platforms. He tweets at @knuutila.
