What you see is (NOT) what you get : semiotics in the digital age

Lavinia Nuvola
Social Media Writings
6 min readOct 2, 2019

--

A photograph, technically speaking, is nothing more than an image created through the act of exposing a photosensitive surface(be it film or an optical sensor) to light. The word itself, photography, is the composite of two Greek words: φωτός (phōtos), “light” and γραφή (graphé), “representation by means of lines” or “drawing” — writing with light. Hence, in a nutshell, we could loosely define photography as an aggregate of visual information — like a text, but with a different alphabet. By clicking the shutter in fact, we do record on a physical or digital surface the memory of an instant; we can record, for example, an event, the emotions of those involved and more holistically speaking, the full context of our lives.

Writing with light: using long exposure times to “see” light
Writing with light. Seili Island, 2015. Photo by the author.

Taking photos has never been easier; if you own a smartphone, a laptop, or a tablet, you’re already able to record snapshots of your life, to retain the memories of a happy afternoon at the beach, or to document the birth of your offspring. The big revolution in the photographic world, however, is not the fact that the camera evolved from being a very expensive and elaborate technique to being a ubiquitous feature in most consumer electronics but the fact that even when consumer photography was already a thing, it wasn’t as public as it is today.

The daguerreotype process. Property of Polytechnic University of Milan
The camera is a relevant feature in the brand new iPhone 11. From Apple website.

Consumer photography before the internet-era, in fact, was relegated to being exhibited in frames hanging on the wall or stored in photo albums proudly showcased to house-guests, while today, our photo album is online and kindly offered for free by big corporations, such as Facebook, Google, Amazon, etc. What was before a private realm, the one of photographic memories, is now out there, for all your friends to enjoy and for all the social media companies to use and to profit from.

The Panzani advertisement analyzed by Barthes. Property of Georgetown University

According to traditional semiotic, and as presented by Roland Barthes in The Rhetoric of the Image, an image is not just about what you obviously see(in the Panzani advertisement used by Barthes, a couple of mushrooms, a few tomatoes, onions and some cheese)but it’s also about the context and how a whole different layer of meaning can be implicitly conveyed or extrapolated. In our Panzani ad, the open bag suggests that one has just come from the market, hence it wants to convey freshness; it has onions, mushrooms, tomatoes and cheese, hence it wants to convey the idea that the Panzani sauce is a complete meal, you won’t need anything else; and last but not least, with the colour scheme and the name of the brand, they want to convey some kind of idea of “italianicity” and all of the mental associations that that brings with it(fresh food, culinary tradition, etc). Of course, in the case of the Panzani brand, it’s a company which tries to convey hidden meanings and context to potential customers; what’s happening nowadays is the same mechanism, but in reverse. Companies are obviously still interested in conveying meanings to potential customers, but they’re even more interested in understanding the context where their potential audience lives from their own photos, in order to tailor better messages and, ultimately, to sell more stuff.

So what does that mean? That when you regularly post your photos online, you might be doing it to share moments and memories with your friends, but companies are using your photos to get more info on your offline behavioural patterns. Photos contain a whole wider range of information about you; what do you wear, where do you hangout, what do you eat, your timetable, your occupation, your relationship status, even though you yourself would not be spontaneously giving that information away. Services such as Google Vision and Amazon Rekognition are specifically tailored to the needs of advertisers and corporations to get commercial insights from your photos.

Promotional materials for Amazon Rekognition technology. Property of Amazon.
Promotional materials for Amazon Rekognition technology. Property of Amazon.
Cameras on HSL bus in Helsinki, Finland, September 2019. Photo by the author.

From this point on, it’d be easy to just go down the dystopian road; and indeed, facial recognition technologies paired with the pervasive presence of cameras everywhere can be daunting; but maybe it’s not all that bad(yet).

Softcover book catalogue of The Family of Man, designed by Leo Lionni, Piper photo by Eugene Harris. From wikipedia.

If in 1955 Edward Steichen needed years to collect 500 photographs from around the world to display in MOMA’s exhibition The Family of Man — with the intent of showcasing humanity in its entirety and diversity, it definitely took less time for Matzen, Bala and Snavely from Cornell University to collect photographic data to publish this study on street fashion worldwide.

And this is to say: there are obvious upsides and downsides to the omnipresence of user uploaded images on social media; on one side, it’s easier for anthropologists, ethnographists and social scientists to study human behaviour, while on the other hand we’re in a bit of a pickle when it comes to social media platforms using our visual data to make a profit.

Matzen et al, 2017. Full study available at https://arxiv.org/pdf/1706.01869.pdf. Property of the authors.

What is the solution, and is there one, really?

What we all know by now, is that we’re giving away a lot of information through the upload of photographs on social media platforms; yet, while societal behavioural patterns have changed already(and it’s just unreasonable to think that people will suddenly stop sharing images online)the legislation, alas, is lagging behind. We’re collectively expecting tech giants such as Facebook, which are operating in a legislative grey area(the internet itself is a fairly new phenomena in the eye of the law)to self-regulate and to behave nicely when it comes to using the information we provide them in exchange for the usage of the service they provide.

The European Union has paved the way for personal data regulation with the GDPR law but when tech giants’ revenue scheme is based mostly on processing users’ data to allow advertisers to better tailor their advertisements, it seems clear that a more ad hoc legislative action would be required from governments. Especially when, according to the GDPR law, only “sensitive” personal information such as ethnicity, health and disability openly displayed in photographs are protected as personal data. So, the whole context of your life is free for social media platforms to use, at least until the law catches up.

--

--