Emojis in Court Evidence

Tara Elonis feared for her life and got a protection-from-abuse order against her ex-husband Anthony after his series of threatening Facebook posts. One post suggested that their son’s Halloween costume include Tara’s “head on a stick.” In his own defense, Anthony Elonis claimed the post was in “jest” because he included a smiling emoji sticking out its tongue. In December 2010, Elonis was convicted of threatening to harm his ex-wife and spent three years in prison. Now Elonis has invoked First Amendment rights to challenge his conviction, causing the Supreme Court to consider what constitutes a threat on social media. Can the little yellow faces that comprise the emoji keyboard really make or break a menacing Internet post or text message?

Internet posts and texts are already a standard form of evidence in criminal cases. But as emojis, sometimes called emoticons, become increasingly prevalent in our textual vernacular, courts still have yet to standardize how to interpret or present the expressive faces and animated props that accompany them.

A slew of recent criminal cases, including the conviction of Anthony Elonis, have sparked new questions about how emojis figure in court evidence. Teenager Osiris Aristy from Brooklyn was charged with making a terrorist threat after a brief series of menacing Facebook posts, such as “Nigga run up on me, he gunna get blown down,” followed by three gun emojis pointing at a police officer emoji. And during the Federal District Court trial of Ross Ulbricht, the convicted mastermind behind the online black-market Silk Road, a virtual Internet bazaar for drug dealers and buyers, a prosecuting attorney sparked debate when he failed to mention the presence of a smiling emoji at the end of an Internet post he read aloud to the jury: “I’m so excited and anxious for our future, I could burst 😄.” After Ulbricht’s defense attorney objected, Judge Katherine Forrest ruled that emojis and punctuation be considered in all evidence.

Unlike in conversation, it can be difficult to gauge tone of voice in text message or Internet exchanges. Emojis offer a way of communicating emotion offering a kind of visual equivalent to tone of voice. But which tone is not always clear.

Whether the prosecuting attorney in Ulbricht’s case purposely excluded mention of the emoji is debatable. Emojis look like something extra, not like real language, said professor John McWhorter, a linguist at Columbia University. “Basically it’s only linguists who know that how you FEEL about what you say is as key to a sentence as WHAT you say,” McWhorter wrote in an email, “most people think sentences are just WHAT you say and the LIKEs and TOTALLYs and WELLs are just ‘junk.’ So naturally emojis seem like the same kind of junk.” But Dr. Robert Leonard, who teaches at Hofstra University, hypothesized that the prosecuting attorney had a strategic reason for averting mention of the emoji. Maybe they wanted to make the evidence seem more “cut-and-dry bad business” than it was, he considered.

No matter the prosecutor’s intention for omitting the emoji, Joshua Dratel, Ulbricht’s defense attorney, maintained that reading textual messages aloud risked skewing the jury’s interpretation. “The jury should read them,” said Judge Forrest. Unlike wiretapped conversations, text messages and the emojis contained within them are not intended for auditory intake. “Chats are designed to be absorbed through reading, not through hearing,” Dratel said in court. Voice inflection and intonation can inadvertently add a layer of meaning.

Meaning itself is a malleable function of the relationship between context and language, which includes emojis. The meaning of an emoticon is contingent on its usage patterns, linguist Tyler Schnoebelen wrote in his Stanford dissertation. Schnoebelen is now the chief analyst at Ibidon, a group that helps companies analyze language data from social media and the Internet. The words that accompany an emoticon, the person using it, and the user’s relationship with the intended audience help define its meaning, Schnoebelen wrote.

The meaning of the emoji in Elonis’ Facebook post is debatable. Contextual information regarding how Elonis generally uses emojis, and how he uses them when talking to or about his ex-wife would help clarify the emoji’s meaning, but not completely. While the presence of an emoji sticking out its tongue could indicate lightness of tone, it doesn’t necessarily indicate a joke, unless Elonis always uses that particular emoji as a way of saying “just kidding.”

Of the 56 round yellow-faced emojis, ten are smiling. The subtle differences among these ten smiling emojis can be hard enough to detect. Contained within a keyboard of 855 characters, the distinct meaning of every single emoji is unsurprisingly murky.

In criminal cases particularly, the jury needs to be educated on what the evidence means, said attorney Derek Shoemake from North Carolina. The jury needs to know the defendant’s intended message. In gang cases for example, Shoemake explained, expert testimony can often elucidate the meaning of slang words. “Emojis are a lot more complicated because people don’t universally understand them.” So far, there is no emoji expert.

Emojis could never have meanings in and of themselves anyways, said McWhorter. Semantics define what you say, McWhorter explained, but pragmatics define why you say it and how you feel about it. And neither semantics nor pragmatics can stand alone. Emojis have to be about something, McWhorter distinguished, “and the something will always be in words.”

The smiley face emoji at the end of Ross Ulbricht’s message — the one the prosecutor failed to mention — would mean nothing without the words that it accompanied (“I’m so excited for our future I could burst”). Emojis add what the text alone is missing: “the FEELING part,” wrote McWhorter. “LOL does the same thing,” he added parenthetically. “They aren’t an add-on, they’re a fill in. They make texts more like actual speech, not less like it.” So when the prosecutor read Ulbricht’s message aloud without mention of the emoji, he left out the symbol indicating the words’ emotional tone.

“It’s the wild west of the emoji era,” said linguist Ben Zimmer. Emojis are so complicated in part because of their emotional value. Facial expressions — in real life and on emojis — communicate emotions on a level language can’t. Emojis can textually express sentiments like embarrassment (😳) that words alone can hardly articulate. But without referencing the “emojipedia,” an obscure emoji dictionary, it’s unlikely the general public would agree on the exact meaning of any particular emoji. In the case of Anthony Elonis’ emoji sticking out its tongue, the meaning is far from absolute. Whether that emoji indicates jest, or disrespect, is up for debate. There’s no piece of language that’s not open to interpretation, said linguistics professor Natalie Schilling of Georgetown University. To attribute standard meanings to particular emojis would be unfair anyways, said Dratel, because we don’t know the author’s true intention.

“I don’t think you’ll ever have a dictionary of these things that people will feel constrained about the same way they do about words,” said Duke University linguistics professor Ronald Butters. Emojis are more fuzzy and negotiable than are words defined in a dictionary. Unless someone gathered thousands of text messages containing emojis to find the most common uses, Butters hypothesized, emoji definitions will never be agreed upon. And even then, interpretation would be loose.

Emojis are right now too culturally specific and even community specific to support standard interpretations of them, Schilling pointed out. A group of people who hang out together might develop their own usages, she explained. Just like any act of language, emojis are a matter of convention.

So when Osiris Aristy added the emoji representations of a gun and police officer to his Facebook post, its meaning was contingent not only on the words accompanying it, but on how Aristy and his group of friends usually used those kinds of emojis. Even though an emoji gun pointed at an emoji police officer could seem like an obvious message at the surface, Aristy’s general pattern of emoji usage informs the tone of his Facebook post. The evidence in his case doesn’t indicate whether in the past, Aristy prefaced his actions with emoji illustrations, but this kind of context would inform just how serious a threat his Facebook post was.

Linguistics and law define threats differently. In linguistics, a “speech act” is an utterance with the intention to commit an act, Schilling explained, such as “I’m going to kill you.” A threat, according to linguists, requires the speaker or writer to believe the act will be harmful to whomever they’re communicating with, said Schilling. They have to intend for the recipient to take them seriously. The legal definition of a threat focuses not as much on the speaker’s intent, but on the interpretation of a recipient who is a reasonable person.

Whether Aristy intended his emojis as a threat, and whether a reasonable person would interpret them as a threat defines the very crux of his case. Speaker relationships are part of the metalinguistic data that give meaning to language — both words and emojis.

Other meaningful metalinguistic factors include intonation and the medium in which the words are delivered. So in the Silk Road case, the difference between reading and speaking is huge, said professor Jill Anderson, who teaches at the University of Connecticut School of Law and completed graduate work in linguistics.

Professor Lawrence Solan at Brooklyn Law School said that emojis are “written reflexes of things like tone of voice.” So, for example, if someone were screaming (indicated by all caps or by an emoji) and the prosecutor read the evidence as a whisper, that would be misleading, he explained.

The textual use of language has a tendency to obscure emotions more than spoken words do. “Emoticons are great because they try to introduce some contextual, emotional or extra information,” said Dr. Leonard. People don’t often grasp how little information is contained in words and grammar alone, Leonard explained. “Context is absolutely everything,” he added.

When he testifies in court as a linguistic expert, Leonard likes to use a seemingly simple sentence to demonstrate the importance of context: “John is on his way to school and is worried about his math lesson.” Who is John? Most people think he’s a student walking to school. Then Leonard adds that last week, his students had a rough time with the math lesson. Now John is a teacher, who most people think is driving to school. Through context, meaning is negotiated and renegotiated all the time.

All too often, the court misses the tone of the text message, a partial function of the texters’ relationship, said Raza Lawrence, a Harvard-trained criminal defense attorney. The text could be sarcastic, an inside joke, or other inside message that’s not readily apparent. Because texts are often misleading, Lawrence and his Los Angeles firm try to exclude text messages as evidence. “There’s so much context within electronic communication that gets lost,” said Lawrence. Texts, including emojis, shouldn’t be admitted into evidence unless both the sender and receiver are present in court for cross-examination, he said. Leonard explained that “you negotiate the meaning with the person you’re communicating with.” When documents are twisted out of context in court, defendants get convicted without the right to confront their accusers, said Lawrence. “It’s been really adhoc. As more communication shifts from verbal to online there have to be new laws or court rules put in place.”

Shoemake thinks emojis are representative of a much bigger problem in courts: social media in general. The Silk Road case was full of provocative technological evidence above and beyond emojis. The court allowed Ulbricht’s OkCupid profile to be introduced into evidence. Based on Ulbricht’s answers to a number of survey questions, the dating site labeled him as more drug friendly than average. People leave important traces of themselves online, introducing yet another challenge for courts trying to determine what to admit as evidence and how. Dratel said another problem is the sheer amount of digital information in a case. “It’s impossible to absorb the entirety of a digital library of someone’s life,” he said.

The Supreme Court says that Elonis’ case “presents a recurring issue of legal and practical importance,” which is one of the reasons they granted his petition. As more communication happens online, the norms for the communication remain unsettled, the court transcript reads. When the speaker’s tone and mannerisms are unknown, online communication is “inherently susceptible to misinterpretation.”

To achieve a standard for how courts treat emojis and social media would require changing the Federal Rules of Evidence, a process that Congress would ultimately need to approve, said Shoemake. That takes time, and social media changes at the speed of light. “It’s hard for the rules of evidence and civil procedure to catch up with social media, particularly in criminal trials.”

Elonis’ case is still pending the Supreme Court, but legal procedure is beginning to accommodate emojis and social media. The smiling emoji in the Silk Road case was not enough to exonerate Ross Ulbricht — he was convicted on all counts, including drug trafficking and money laundering. But requiring emojis to be considered as evidence in the Silk Road case has opened up a new era in the treatment of online or textual evidence: prosecutors have since acknowledged emojis.