MLUX Data and Policy Panel July 2020

Governments at all levels are being called on to institute rapid policy changes in response to the global pandemic and demands for social justice. Moreover, 2020 is a US presidential election year and the year of the US decennial census. Crucial initiatives rely on interpreting data or developing models that inform policymakers. But with concerns around privacy and the significant risks that new technologies introduce or exacerbate, there are a few key elements that today’s citizens should be familiar with in order to contribute to policy debates.

MLUX’s Anmol Anubhai sat down with Elizabeth Adams and Amy Smith to discuss this timely topic

Elizabeth Adams (@technologyLiz) is an Expert Advisor for the United Nations Global Advisory Body on AI Cooperation and a Stanford University Race & Technology Fellow. She works at the intersection of Cyber Security, AI Ethics and AI Governance, focused on Ethical Tech Design. Elizabeth is also an appointed member of the Racial Equity Community Advisory Committee for the City of Minneapolis influencing the local Civic Tech & Tech Design Racial Equity conversation and framework.She passionately teaches, advises, consults, speaks and writes on the critical subjects within Diversity & Inclusion in Artificial Intelligence, such as racial bias in Facial Recognition Technology, Video Surveillance, Predictive Analytics and Children’s Rights.

For the past decade, Amy Smith ( @amymapsmith) has been working at the intersection of mapping, transportation, technology, and public policy. Most recently she worked at Uber as a Data Scientist on their Policy Research & Economics team, where she studied the impacts of new mobility services on cities around the world. She’s currently a Senior Data Scientist at Rebel Group, an international consulting firm focusing on sustainable urban development and mobility.

Data informs policies (and that’s generally a good thing) but how does it work?

It starts with a question

According to Amy, there is not one formula that applies across the board. Often there is a question that starts the process like: what will be the impact of COVID19 and public transit? That’s a really big question, so she starts by trying to get as complete an understanding of the question as possible. Desk research, talking to stakeholders, and speaking to people outside of her bubble to find her blind spots all help to refine and break down the different facets of transit and mobility that are relevant for policy makers and communities.

Then find relevant data

After a period of time working in a particular domain, you will develop familiarity with a list of go-to data sources that you trust — or at least sources for which you understand the strengths and weaknesses. For transportation Amy’s go to list includes the following, though these sources cover a broad range of useful insights:

• US Census Bureau and American Community Survey
• National Household Travel Survey
• EPA Smart Location Database
• World Bank
• ILO
• IMF (COVID-19 Specific)

You may also be able to find open data that local communities or companies publish. For example, Uber may share some of their data with researchers at universities. Especially in light of the global pandemic public officials and companies are compiling and sharing public health relevant data sets. Apple for example created mobility trends reports as a response to COVID19.

Big caveat! Be cautious when using data that was collected for a different purpose from yours. You may not know what population was sampled and what biases exist in that data (not to mention any ways that it has been ‘cleaned’ or transformed prior to you seeing it). To assess the trustworthiness of data, here are a few good questions to ask:

• Who is represented in the data? Who is NOT represented in the data?
• If the data is from a survey, what’s the sample and time frame, and could the purpose of the survey and/or the way the questions are asked potentially create bias?
• If the data is crowdsourced, is there any information on accuracy and/or quality of the data?
• Is the data a synthesis of other datasets, and if so, is the methodology and source data sound?

Be open and direct about any assumptions. Especially in the post-COVID world, we may not be able to rely on historical data anymore. If the past behavior is not predictive of future behavior now or when things are “normal” again, it’s important to communicate those uncertainties. Then, if you can, test scenarios based on our assumptions to get a more holistic picture of what the future might look like.

Sometimes the data doesn’t exist at which point, you have a couple of choices. Best case scenario you can conduct primary data collection for your purpose. When that’s not a possibility, consider the use of proxies (though see warnings above!). And when you really come to a dead-end, Amy points out that this is also a very interesting question in itself: Why is that data missing? And what are the policy-relevant questions that we’re not able to answer because it’s missing (or what policies lead to or prevent its collection?) And can we do something about that?

Consider the impacts, especially second and third order effects

This may be the most important and most often overlooked step of the process of inclusive policy development: beyond the initial outcomes, what other effects may come as a result? For example, how will it impact marginalized communities? Elizabeth highlighted for us the challenges, questions to ask, and some tactics to improve inclusivity.

In her past, Elizabeth just had to trust those who were designing and building technology to think of ethics in their process. But after a personal incident and some perspective, she realized that she needed to better understand AI.

She recommends hosting learning events as both a good way to start educating yourself, but also as an approach to align with other people at your organization. In hosting learning events, Elizabeth was able to crystalize her perspective and gain confidence in sharing her beliefs. The biggest learnings come from bringing in real life examples of what happens when ethics and transparency doesn’t exist to help highlight the second and third order effects. Though we could provide hundreds of examples of issues, the phenomenal work of Joy Buolamwini and Cathy O’Neil is a good primer. Elizabeth suggests these questions to form your perspective and when developing technology or policies:

• What does fair and equitable technology mean?
• What does safe technology mean?
• What do these mean for communities of color and vulnerable populations?
• Do you understand what happens to the technology that you build once it’s out in the marketplace?
• Do you understand if the people that will eventually use this technology have received enough training to use it properly? Have they been trained to identify bias?
• And do you understand what actually happens to the real world communities that are impacted by that technology? (especially consider the representation of potentially impacted communities in your data and policy development process)

It is clear we must know who is designing our public policies, what decisions have to be made about those policies, and most important, to understand what will and does happen to real people when policies are implemented. Even more so when policies are using new technologies.

Get more involved with your community policy makers

Elizabeth’s advice:

1. Start with an internet search on boards and commissions in your city and/or state. Here’s an example for MN and here’s an example for Minneapolis
2. Find one that you are passionate about and don’t get discouraged if there isn’t one that aligns with technology directly. This is key, I am able to influence the Civic Tech space through a Race & Equity lens. There was not a technology board. What was closest to my passion was racial equity? I have a colleague who sits on the MN Game and Fish commission. She wanted to improve participation and opportunities for women. She loves to fish. She’s been successful in helping the state dedicate resources to training/jobs/events for women in this space.
3. Reach out to the Council Members or Representatives in your ward. Most love when the community steps forward to help them. My connections and influence are invaluable.

About the Machine Learning and User Experience (“MLUX”) Meetup

We’re excited about creating a future of human-centered smart products, and we believe the first step to doing this is to connect UX and Data Science/Machine Learning folks to get together and learn from each other at regular meetups, tech talks, panels, and events.

Interested to learn more? Join our meetup, be the first in the know about our events by joining our mailing list , watch past events on our youtube channel, and follow us on twitter (@mluxmeetup) and Linkedin.