MetaMetrics Interview #1 — Bobby Dresser
These interviews are part of a deep-dive series that hopes to elucidate some of the thought processes behind MetaMetrics in a more casual manner. If you are looking for our more compact announcement, please check here. Interviews have been edited for readability and brevity.
Bobby Dresser is a Product Manager at MetaMask who has been on the team for approximately a year. He has previously worked on another project within ConsenSys called uPort before joining us.
Interview With Bobby Dresser, Project Manager
Kevin Serrano: We recently announced our new analytics system, MetaMetrics. MetaMask hasn’t been doing much in the way of analytics for years. Why start now?
Bobby Dresser: If we want MetaMask to be successful for a broad set of users beyond a technical crowd, we have to understand how people are actually using the product. We can’t just build with our own intuition or what people on GitHub tell us. It will be helpful to understand which MetaMask features people use, what aspects are confusing, and what people spend a lot of time on. Although qualitative research goes a long way, it doesn’t quite give us the detailed picture that we need now.
Kevin: Could you tell me a bit of background as to how this project came about? Who instigated it, how did it gain traction, etc.
Bobby: It’s something we’ve talked about for a while. There’s obvious hesitation given that our users are very privacy conscious. We want to make sure this happens in a way that is very detailed as to what information we’re collecting and when. We are making it entirely opt-in; if you don’t want analytics events to be sent, then you can entirely opt-out. We’ve wanted the data for a while, but there’s so many different things for us to do as an organization that it’s been hard to prioritize this.
Kevin: How does the team reconcile gathering data from our users? You mentioned we have this commitment of respecting privacy. How do we reassure users that we still value their privacy?
Bobby: Making it totally opt-in. We’re not selling anyone’s data for revenue. We’re trying to understand how people use and don’t use MetaMask so that we can build a better product. If you’re concerned about sending these events, then you don’t have to send them. If you do want to help us build a better product that works for different types of people, then you could consider enrolling. We will also do basic procedures like obfuscating IP addresses, allowing people to opt out even after they’ve opted in, and retroactively deleting all of the data that we’ve gathered in our analytics service.
We’re using an open source service called Matomo which is pretty widely and positively regarded as a solid tool for this purpose. We also want to publish a subset of this data so that other developers in the community and people in general can understand Ethereum usage patterns.
Kevin: Given that we are collecting this data, is it still within our ethos to release that data as something for the community to benefit from?
Bobby: Two things. First, we’re not going to say, “people in Massachusetts like to add more than 30 different tokens.” We’re going to say, “this is how many monthly active users we have.”
Secondly, we have a philosophy on the team that anything we know about the product, that is good enough for other people to know. All of our production, all of our future work, all of our bugs, everything is publicly tracked on GitHub. If we’re going to collect data and understand how people use the app, then other people should be able to see that data. I don’t think we are privileged enough such that there is knowledge about the product that our wider user base isn’t allowed to access.
Kevin: From your perspective, what kinds of questions and insights do you hope we’ll be able to answer with this new information?
Bobby: I think there’s a lot. Part of it is about trends and usage such as: we know how many people have the extension, but how many people use it? How many people open MetaMask to track their own tokens versus use it to interact with Dapps?
Often, it’s hard to reconcile different things that people want. Someone will say, “I want a button for this!” And someone two weeks later will say, “take away this button, it’s confusing.” Right now, we don’t have data other than this qualitative source of input from our support channels and from GitHub. We don’t have a solid understanding of the question, “which of those people speaks for a broader portion of our user base?”
I think also we have a general understanding of what’s easy and what’s hard for users but we can get a lot more specific with that. For example, where do people drop off because the experience became too impossible to navigate? By observing some of those funnels, I think we’ll see with great detail the biggest adoption hurdles.
Kevin: Is there a way for us to allay fears that this is not a slippery slope?
Bobby: I think it’s a basic question of trust. It helps that this code is all public so anyone can search the code base for specifically what analytics events were firing. Additionally, you could ask the same question about why you’d ever trust this thing to hold your private keys and how you know it’s not phoning them home. So if we explicitly say that we will never collect addresses, keys, and balances, that word is as binding as our code.
Kevin: Then for my last question, is there anything else you’d like to close off with?
Bobby: There’s a general balance between purity and mass appeal. What we’re trying to do is move towards mass appeal and broader usability without sacrificing fundamentals such as privacy, security, or trust, and introducing metrics that help us tune the application so that we meet users where they expect to be met.
At the same time we want to maintain everyone’s trust. We want to make sure that people who are incredibly privacy conscious don’t have their experience affected at all. That’s the basic bargain of what it means to use the decentralized web and it isn’t transcended by our desire to be more usable.
