MetaMetrics Interview #2 — Dan Miller

These interviews are part of a deep-dive series that hopes to elucidate some of the thought processes behind MetaMetrics in a more casual manner. If you are looking for our more compact announcement, please check here. Interviews have been edited for readability and brevity.

Dan Miller is a Full Stack Web Developer at Kyokan, a contracting team that helps us build MetaMask. He has worked on the MetaMask project for approximately one and a half years.

Interview with Dan Miller, Engineer

Kevin Serrano: Given that you’ve been here for awhile, you know we haven’t had an analytic platform for a long time. Why is it necessary for us to develop and implement such a suite now?

Dan Miller: Ever since I’ve been part of the project, MetaMask has been constantly faced with the challenge of trying to figure out how to build effective user interfaces for the blockchain. A big concern here is how to make interaction with the distributed web accessible for new people, for whom there’s so many new concepts. On top of that, we have to balance that challenge with the need to support a large community of developers that have come to rely on MetaMask. That’s a range of user experience concerns that are really hard to solve.

The only way you can do that effectively in the long term is by making decisions based on sound data. MetaMask does a good job trying to understand their users from lots of angles, but quantitative data on actual user behavior within the app is critical.

Kevin Serrano: Can you give some technical insight as to how we would retrieve the data? What kinds of data points are we collecting?

Dan Miller: Our initial focus is on understanding what users do. What actions do they do? Not do? There’s also understanding the steps users take to arrive at a specific screen or complete a specific task within the app.

To measure all those actions, we send an API request to a special endpoint called Matomo, an open source analytics platform. We can send a GET or a POST request with any range of different query parameters that define the data that Matomo collects. There’s lots of custom data that we can collect: we can label the action and we can categorize it. We can also specify the page on which the action happened. Matomo then accumulates the data and gives us a nice platform through which to understand the data.

Kevin Serrano: So the data being collected is specifically user actions? Does it include anything like a user’s personal information like their accounts?

Dan Miller: Matomo can collect a lot of data apart from specific events. Matomo can collect things like IP addresses, browser type, and possibly operating system type. Those are all things Matomo can collect by default by the fact that user clients are sending API requests for them.

Now, we can turn off a number of those things. We’ve made a decision not to collect IP addresses, geographic data, account addresses, or transaction hashes. There won’t be any way to identify any actions an actual user takes.

We made a decision to associate an ID with users between sessions so we can get longitudinal data. When that shows up for us, it will be nothing more than seeing that a user who:

• Last time, sent two transactions last time and then hit an error on the customize screen
• This time, didn’t use the customize screen at all.

There won’t be any way to tell who that user is. No accounts. Nothing like email or anything like that.

Kevin Serrano: How are we confident that this data is safe? How can we be assured that Matomo is a reliable place for us to keep this data?

Dan Miller: We chose Matomo because they are extremely explicit and committed to privacy and about MetaMask’s custody of the data. It’s an entirely open source platform, so anyone can audit the code that it’s operating on. In addition, all of the API calls will be fully transparent to the user through their browser tools.

The risk we run is the same risk anyone runs when you’re storing data on the Internet: we can’t assume that Matomo’s servers are bulletproof, although they have a very strong track record with regards to their hosting service and keeping user data safe.

Kevin Serrano: Could we expand a little bit more on exactly how the data will be viewed by our design and development teams?

Dan Miller: All these events that we’re tracking within Matomo, we’ll be able to look at those on an event-by-event basis and see raw numbers on how many users are doing them. We’re going to create within Matomo what it calls a “user flow”. We associate a handful of events and then see how many users complete those events in sequence, whether to completion or not. We can also associate broad groups of data presented to us in table form or charts and graphs. We can ask questions like, “how many users, after sending a transaction, open a link to Etherscan?”

Kevin Serrano: Two more questions. First, what are you excited about? What do you feel like this will enable us to do in a timely manner?

Dan Miller: One thing that we’ll be able to do with a lot more confidence is experimentation. Not to say that MetaMask is not good or already great, but we’re still early in development. There’s many ideas for how certain concepts could be improved, but actually selecting those ideas and rapidly testing them out is a limited resource.

Some of this is inhibited by doubts about how many people would this matter for. Will this frustrate anyone or will it be a good change? With the analytics, this unleashes a lot of those inhibitors to experimentation and allows us to get more quickly from maybe to certainly.

Kevin Serrano: Is there a specific feature improvement or implementation in mind that you’re hoping to experiment with first?

Dan Miller: The one that comes to mind first is the first time flow. We know that the first time flow is a big inhibitor for lots of new users. How is that going for users right now? What are the key points of friction and let’s get a few different ideas out there and see how that would improve things.

The other thing that I’m interested in is how users navigate between screens. What do users do when they get to the main screen? What do they do first? If we understand that, we’ll be able to help a number of users speed up their workflows.

All of this will also allow us to identify new ways that users can customize their experience. I can see a future in which different sorts of users have access to different sorts of user flows, because there’s clearly distinct user segments that wish to take advantage of different user flows. When we get there, we’re in a really strong place to make crypto technology accessible to the masses. We also want to build this powerful platform that devs can use to make whatever they’re building accessible to the masses. Our ability to customize and understand any key user segments allows us to do that much better.

Kevin Serrano: Any last words?

Dan Miller: No, the only other thing I can say is as a developer who will be handling a lot of the MetaMask side of this integration: I am committed to user privacy and ensuring that we fully honor our users’ trust in us. Anytime I work on analytics, I’m going to bring that to the table 100%.