Medicus Sunday Serial: Maps, Tweets and Hippocrates, and the role of ethical design
Ethical design can be seen from several angles, but the conclusion is always the same: “Do no harm”. From Google Maps’ core features, to Twitter’s strict no-editing stance, to the role of philosophers in modern apps and AI-driven technology, ethical implications arise.
Read Part 1 here if you prefer to prepare for the sequel, or dive directly into Part 2 below.
Serene Touma, Head of Content and Marketing, Medicus AI:
Let’s dive right in! In your opinion, which historical product do you think has had the big biggest ethical impact on the way we live and work today?
Makram Saleh, Chief Product Officer, Medicus AI:
A good example of something very current is map and directions apps. The issue here is that directions generated by technology are able to route you through roads with less traffic which introduces a dilemma that the original designers of a service like this most probably never thought about. Routing a user through small neighborhoods to avoid main congested roads brings unwanted traffic to areas that didn’t have it before, turning quiet neighborhoods into traffic nightmares.
This is the kind of thing to think about as a designer. You have a great service, doing what it is supposed to be doing, and yet the result is unforeseen problems that are harder to mitigate after the fact.
[There’s an artist that took this feature to the extreme and tricked the system into creating traffic jam alerts.]
Serene:
Absolutely. These are the kind of unintended consequences that could be easily written off as non-malicious and therefore irrelevant to the question of ethics, but a closer look uncovers an ethical implication. To what extent do you think the question of ethics should come into play at the design stage?
Makram:
We design services to fulfill certain requirements, or to provide a useful option, but sometimes this results in unforeseen outcomes. A great example is the “avoid highways” option in Google Maps, which is objectively useful and unproblematic, much like the “avoid traffic” features, which is an expectation in any good routing software.
So while designers thought they were doing a “good” job, they were actually facilitating an ethically ambiguous outcome. In this case, who is to blame?
If their needs are not met, users impose pressure by leaving the app or service because they feel that it doesn’t fulfill their expectations. Are designers supposed to act as the gatekeepers here?
Serene:
The ultimate rhetorical question. We sometimes see the opposite extreme where designers find themselves playing it too safe; not including something that could be useful, just because it might be interpreted as having a malicious aspect to it. What we have to consider is how to mitigate any consequences, and this is where designers come in.
Makram:
Exactly. I’m not saying designers should just shrug their shoulders saying “we did our best, take it or leave it.” That kind of unforeseen problem most often requires designers to go back to the drawing board and consider how to mitigate it.
Even when something “just works”, responsible designers are still looking for potential issues that might result from their designs.
They are carefully observing how their designs are received by the intended audience and making changes towards better user experience. In that sense, the design job is never finished. Responsible companies and responsible design teams do this all the time.
Serene:
We are in the healthcare space where the Hippocratic Oath is commonplace: “do no harm”. It can be viewed as a framework and a reminder of where our responsibilities lie. At Medicus, we work in such close proximity to doctors which has created that space for us to also think in this way, so I can see how designers in transportation or e-commerce may not immediately consider these factors in the same way.
Makram:
Speaking of Hippocrates… in every domain, and whenever there is a breakthrough at the boundaries of that domain, whether socially, technologically, or culturally, there are always philosophers involved. You start thinking about the bigger questions. What’s the impact on society? What are we trying to achieve here?
Serene:
This is a fascinating point and it reminds me of an article I read recently about which jobs are going to die out because of technology and which ones are going to grow, and the one that was at the top of the list was philosophers.
It’s been hundreds of years since philosophy was something that was regarded as essential to the development of technology.
With AI and with algorithms comes a lot of responsibility so it’s very important for us as a culture, but also as a company, to have a philosophy that we agree to as our North Star.
Makram:
I see that the conversations between technologists, designers, and developers on one side and philosophers on the other as being something very, very important. Some people might dismiss that kind of dialogue claiming that philosophers have limited knowledge and impact on technology, which, I think, is not helpful.
Contemporary philosophers like Sam Harris, for example, are playing an influential role, bringing important questions to those conversations. He’s aware of the concerns and implications of technological developments and brings them up in eye-opening discussions with influential people, like this conversation with Jack Dorsey of Twitter.
As designers, we should educate ourselves at this level, even if that kind of education is not directly relevant to our day-to-day jobs, especially if we want to tackle big problems. Otherwise, we’re stuck working on small, easy problems.
Serene:
Jack Dorsey is someone who has really stood his ground on a feature that he has been asked over and over to implement, which is the ability to edit tweets. It seems benign on the surface: “so what if people can edit their tweets?” but it is a big deal.
I respect his decision to stand by this because I see it as a seemingly small but very concrete way for him to say, “This is an ethical line that we will not cross.”
Makram:
Yes, it’s the things that may seem small, but are actually quite important to debate because they set the tone for a lot of other things. This brings me back to the point we started talking about earlier, that just because a designer can do something, it doesn’t mean they should. From the early days of forum comments, implementing comment edits was always technically feasible and almost obvious, but it was really a lightbulb moment when we stopped to consider the implications of a comment edit or deletion. Thinking a couple of steps further, it’s easy to see that it’s not simple: if a comment enables the conversation then it’s part of a content ecosystem, it’s not independent. Editing or deleting a comment can affect context and influence meaning.
Serene:
You’re speaking music to my ears as a writer because, as you know, we are precious about our words and feel a very strong sense of ownership about them. So when it comes to ethical concerns, who owns the content? Do you think there’s an ideal?
Makram:
In my opinion, there is no gold standard. Are we going to have three philosophers sitting around the table to decide on each and every change or design decision? What makes sense for me is to think systematically and focus on building the right culture in which design decisions are considered and made.
Serene:
This takes us quite nicely to my last question. From your perspective, what do you think the biggest philosophical debates are for us at Medicus? In other words, what are the things that you think are going to have the greatest impact on our users that we should be thinking about today?
Makram:
I wouldn’t say I’m not sleeping nights because I’m always worried about this. But this topic of brushing against ethical issues comes up often and in unexpected ways.
For example, sometimes in meetings with a big client or healthcare provider, requests come up which may not necessarily be unethical, but I would say not thought through to their conclusion. They are simply asking for requirements that enable their business. This is where the ethical design mindset needs to kick in.
A lot has been said and written about technology enabling insurance companies to discriminate against insured patients, and this is something that is on our radar. Obamacare is an example of an ethically designed program that made sure insurance companies did not discriminate against patients with pre-existing conditions. Protecting patients’ personal data is extremely important in this regard.
For us at Medicus, privacy has been a priority from day one when designing, it’s not just a buzzword. Our challenge is to enable rich features for clients like insurance providers without jeopardizing patients’ privacy, for example, we design technology that allows us to detect the risk of certain diseases across a large group of people rather than in an individual’s profile.
Thinking about these ethical dilemmas and having a clear stance about big issues is what differentiates a good designer from a great one.
Serene:
We could talk for hours on this topic, and I’m sure we’ll talk plenty more, but for now, I want to say thank you for taking the time to have these discussions with me.
Makram:
And thank you for posing the questions! As designers, we should educate ourselves at this level, even if that kind of education is not directly relevant to our day-to-day jobs, especially if we want to tackle big problems. Otherwise, we’re stuck working on small, easy problems.
To receive stories and conversations like this periodically in your inbox, sign up the the Medicus Journal here.
