A Case Study on User data
User data collection is shaping up to become one of the largest markets in the world. Wherever money goes, ethical quandaries are often not far behind. We hope to shed some light on how quickly the industry, why these topics matter, and hopefully stimulate debate with some questions that have already surfaced.
Why does this matter?
It’s no mystery that this is a booming industry. As of late, it’s becoming more and more mainstream. Many things have helped with that. Facebook’s debacle with Cambridge Analytica, Russia’s election interference, and too many hacks to count. All of these events have brought the spotlight onto how important information security is, and how important it is too have a conversations about who gets to see what information.
Case:
A marketing company wants to understand more about the social behavior of a community. The marketing company suggested to a popular phone manufacturer to identify ever phone with a particular user and to have the phone collect data constantly. The type of data that would be retrieved from the phone are the following: GPS location, Time, Call History, Text Messages Count (Not the actual text of the messages. Just the number of messages sent and messages received). On top of that, all this data would be related to a user profile within a database that contains sensitive data such as gender, race, age, sexuality, and potential medical records. The phone company would be paid with a large sum of money and the reason for this is that the marking company wants to create an AI model that takes this data and uses it to learn about people habits. From there, the AI would be used to understand when people like to do certain activities at certain times so that the marketing company can consult with local business and enhance their businesses.
Breaking Down the Case
To tackle this case, we’re going to approach it with some questions we beleive should be kept in mind.
The purpose/intended function of using computing in the hypothetical scenario.
This application of technology is already happening in mass around us. We’ve all had tailored ads that simply won’t leave us alone or ads different from friends with similar interests due to age, race, or gender. This case posits the situation of going further than this however. This marketing push would be, in many ways, invasive. It would attempt to create AI that would accurately predict your daily routines and when you might need something before you even know it. Past that, this information could then be brokered back into the local economy, enhancing their ability to accurately predict what stock they may need and better cater to the community.
The various types of stakeholders that might be involved in such a practice, and the different stakes/interests they have in the outcome.
A system like this could help, and potential harm, many different types of people. First is everyday people. They certainly have the most to lose in this situation. Any breach in this taken data would be devastating for them. Even if there is an opt-out feature for this, many people may still unknowingly be affected, which would be awful. The depreciation of privacy is also concerning. However, they do benefit from the stimulated economy as well as enhancing consumer experience. The phone company oddly has no nearly as much to gain as one would think. Besides cash, the phone company become the subject of potential cyber attacks and ramifications if things go awry. The local businesses would likely see a great increase in their margins, however, forming a reliance on another company is a dangerous proposition. It also puts any companies that don’t participate and use the marketing service at a distinct disadvantage. We also must consider the most likely to prosper greatly from this endeavor, the marketing company. As previously mentioned, information is a booming market.
The potential benefits and risks of harm that could be created by such a project, including ‘downstream’ impacts.
The way in how this phone is implemented is important. This will cause a reaction to the consumer community. If the phone manufacturer is open about how data is being scraped from the user at all times, the phone manufacturer may receive bad press. If the phone manufacturer receives bad press, then this could drop phone sales which in turn drops down the amount of data that is collected. A questionable route that can be taken by the phone company is to leave this hidden until further notice. That is, at the launch of the phone, do not tell anyone about the AI Model that is in development. From there once the AI model is fed data and can produce benefits for consumers, then let the users in on what occurred. The benefits would outway the deception of the company. Also, the consumers are already using the phone so it is more likely for the consumers to accept what the company has done.
The ethical challenges most relevant to this case
There is a problem with the unwarranted collection of data done by the phone company with this scenario at hand. The people who pay for the phone are becoming machines that produce data for a purpose of monetary gain. At this point, the company is not giving the consumer the opportunity to understand what is fully going on with the product they are buying. To move forward with is conversation is to consider the control and rights that the consumers have on the data they create with consumer company products. Does there needs to be a line of communication to the end user that informs them that data is being taken from the them. Does there needs to be a set of rules that allow the user to opt out of some, none, or all forms of data collection. It is important to understand that although the user is giving data to the Phone company which in hand is giving the data to the marketing company to create an AI model, the user will benefit from the marketing companies implementation of using the AI model to enhance consumer life. On top of that, we must be reminded of the freedom that is the internet. The user only needs to pay an Internet service provider to be connected to the internet. The user does not need to pay facebook to be able to go to facebook’s website. But facebook needs to pay all their engineers high salaries to be able to engineer facebook’s application infrastructure. In the digital world, the end user can not have all the power. The user must choose between the red and blue pill. Does the majority of users want to pay for individual applications that are provided on the internet so that they never collect data for monetary gain or does the user want to give up their data in exchange for using application for free. There is a fight for control and against the user and application provider.
One way that the risk of the worst-case-scenario could be reduced in advance, and one way that the harm could be mitigated after-the-fact by an effective crisis response.
This is a scary situation to imagine and putting boundaries in place is important. We believe that one way we could prevent a situation from this every happening is to create a task force, either within or outside of the government, whose sole responsibility is to analyze every user data broker and decide whether a line has been crossed. Keeping this party unaligned with companies would be the difficult part of the plan. One way to navigate this is to have potentially many smaller, local bodies that are elected by popular democracy as opposed to a representative to ensure they have the people interest at heart.
A worst case scenario is that we introduce a biased or simply incorrect AI model and have it truly affect the population. If this were discovered after its already occurred, the best recovery would be to obviously stop using the system as soon as possible. Correcting the system is on possibility, but at that point it might as easily swing in the other direction. More likely, the best outcome is to help the group that was damaged, find out why the system didn’t work, and ensure it doesn’t happen again. This is a big issue with why these system are so dangerous. Many times the damage done is subtle but consistent and there’s little to do to build back what was lost.
In conclusion, this is a system that has many drawbacks, but it isn’t all negative. Many great things can sound negative on paper, such as nuclear energy and paperless economies. While the strengths listed are few, many of them would have profound, widespread, positive impact on many people. We simply believe caution is always advisable.
Bibliography
- Santa Clara University. “Removing a Search Result.” Markkula Center for Applied Ethics, www.scu.edu/ethics/focus-areas/internet-ethics/resources/removing-a-search-result-an-ethics-case-study/.
- Santa Clara University. “Targeting a Broken Heart.” Markkula Center for Applied Ethics, www.scu.edu/ethics/focus-areas/internet-ethics/resources/targeting-a-broken-heart/.
- Santa Clara University. “Data Collection: ‘Harvesting’ Personalities Online.” Markkula Center for Applied Ethics, www.scu.edu/ethics/focus-areas/internet-ethics/resources/data-collection-harvesting-personalities-online/.
- Granville, Kevin. “Facebook and Cambridge Analytica: What You Need to Know as Fallout Widens.” The New York Times, The New York Times, 19 Mar. 2018, www.nytimes.com/2018/03/19/technology/facebook-cambridge-analytica-explained.html.
- Santa Clara University. “China’s Social Credit Score.” Markkula Center for Applied Ethics, www.scu.edu/ethics/focus-areas/internet-ethics/resources/chinas-social-credit-score/.
