Digital Ethics: Whose responsibility is it? (2/3)

UX folks may be in the best position to identify ethical issues in their companies. Should it be their responsibility?

In the previous section, I described the state of UX practice at technology companies, and the need for high-level buy-in for successful UX integration.

There is a concerning — and increasingly evident — lack of ethical consideration in the processes of most software companies. In this section, I will describe some of the ways in which this has recently become more apparent.

Digital Ethics

The software in our lives are not generally designed with our health and well-being in mind. This fact is becoming clear as Facebook, Google, and Twitter are in the spotlight relating to Russia’s interference with our elections and increasing political divides. Twitter has also typically been unwilling to do much about threats or hate speech.

There is too much focus on engagement and creating addiction in users, and not enough on how things might go bad and appropriate ways to handle that.

Internet of Things (IoT)

There’s a proliferation of products in the Internet of Things (IoT) space, many of which are completely insecure and thus easily turned into a botnet, have the private information on them exploited, or hacked to be used as an information gathering device.

Effects on Kids

Some IoT devices are specifically targeted at kids, but few or no companies have put any effort into identifying how they will affect the development of the children who use them. Concerned researchers at the MIT Media Lab have begun to study the effects of intelligent devices and toys on kids, but this won’t stop the continued development of these devices.

Similarly, it’s unclear how the use of devices that were originally aimed at adults — such as Alexa — will affect the kids in those houses. On one hand, it doesn’t involve screen time, which is no longer completely contraindicated for kids under two but is still wise to limit. On the other hand, we have no idea how those devices will answer questions they were not programmed to handle. Additionally, these devices do not encourage kids to use good manners — one of the important lubricants for the fabric of society. It’s hard enough to teach kids manners without having that teaching undermined by an intelligent device!

Finally, consider how machine learning can result in some truly horrific scenarios (content warning: the linked essay describes disturbing things and links to disturbing graphic and video content).

Someone or something or some combination of people and things is using YouTube to systematically frighten, traumatize, and abuse children, automatically and at scale, and it forces me to question my own beliefs about the internet, at every level.
James Bridle · Writer and Artist

Willful ignorance: Twitter and Equifax

Similarly, we’ve seen the results of a focus on metrics and money over security and sanity. Twitter not only knew that there were spam and fake accounts from Russia and the Ukraine in 2015, but refused to remove them because

“They were more concerned with growth numbers than fake and compromised accounts,” Miley told Bloomberg.”

Equifax stores highly sensitive information about people in the US, and left security vulnerabilities open for months after being told about them. As a result, they had multiple security breaches, basically screwing over anyone whose data was stolen.

Yeah, no. You knew you had vulnerabilities!

Thoughtlessness: Google, Facebook, and Big Data

Even without willful ignorance, thoughtlessness alone can easily be enough to put individuals, communities, and societies at risk.

Considering the breadth of data that many companies are collecting on those who use their products, there is a worrying lack of thought given to the invasiveness of this practice and to how to safeguard the data in question. These companies often make poor choices in what information to keep, how to secure and anonymize the information, and who has access to that information.

Some might say that having conversational devices like Alexa and Google Home are worth the privacy risks inherent in an always-on listening device. Others might suggest that it’s already too late, given that Siri and Google Now have been listening to us and our friends through our phones for a long time now.

However, regardless of one’s thoughts on the timing of the concerns, the fact remains that tech giants have access to an amazing amount of information about us. This information is collected through our phones, through our searches and purchasing patterns, and sometimes through devices like the Amazon Echo and the Google Home Mini.

Some companies are better than others, such as Apple’s refusal to break their encryption for the FBI, but it can be quite difficult to identify which and where companies are making the best choices for their customers privacy, safety, and sanity.

Machine Learning

Take machine learning (also known as AI), and the fact that companies are more interested in selling ads than considering the effects their software has on their customers:

It’s not that the people who run, you know, Facebook or Google are maliciously and deliberately trying to make the country or the world more polarized and encourage extremism. […] But it’s not the intent or the statements people in technology make that matter, it’s the structures and business models they’re building. […] Either Facebook is a giant con of half a trillion dollars and ads don’t work on the site, it doesn’t work as a persuasion architecture, or its power of influence is of great concern. It’s either one or the other. It’s similar for Google, too.
Zeynep Tufekci · Techno-sociologist

One of the major problems with machine learning is that we have _no idea_ precisely what associations any particular algorithm has learned. The programmers of those algorithms just say whether the output those algorithms provide is good enough, and often ‘good enough’ doesn’t take into account the effects on individuals, communities, and society.

I hope you begin to understand why ethics is a big concern among the UX folks I follow and converse with. At the moment, the ethics of digital products is a big free-for-all. Maybe there was a time when ethics wasn’t as relevant, and code really was just code. Now is not that time.

In part 3, I’ll discuss the positioning of UX people to more easily notice these issues, and the challenges involved in raising concerns about ethics and ethical responsibility.

Thanks to Alex Feinman, Máirín Duffy, and Emily Lawrence for their feedback on this thread!