MetaMetrics Interview #3 — Omna Toshniwal & Christian Jeria

These interviews are part of a deep-dive series that hopes to elucidate some of the thought processes behind MetaMetrics in a more casual manner. If you are looking for our more compact announcement, please check here. Interviews have been edited for readability and brevity.

Omna Toshniwal is a Design Researcher at MetaMask who has been on the team for just under half a year. Christian Jeria is our Head Designer at MetaMask who has been on the team for just under two years.

Interview with Omna Toshniwal, UX Researcher, and Christian Jeria, Senior Designer

Kevin: Christian, you’re probably more suited for this first question. Given that MetaMask has gone this long without doing significant analytics, why is now the time?

Christian: That’s a great question. I think we are at an interesting time in our ecosystem. In 2018 Blockchain reached a level of mainstream no one ever expected. Since MetaMask is an integral part of onboarding to using Ethereum and Dapps, I think it’s the right time, from a purely design research perspective to introduce analytics so that we can better understand users pain points and build better features and products.

Kevin: Omna, would you like to add anything onto that question?

Omna: This is a great opportunity for us to understand where people stumble. How can we make this experience easier for people to learn? We want to explore how we can make MetaMask more friendly for people new to blockchain, while we continue catering to the needs of the early adopters.

MetaMask has been around for a few years now and has reached a decent amount of users. MetaMetrics can help us understand at scale how people are using MetaMask.

Kevin: What are some of the high level questions that you’re trying to answer through these new metrics?

Omna: Some standard ones are: how many people are using MetaMask? Which features are they using and not using? What kind of paths do people take to reach a certain point? Can we improve discoverability? Analysis and aggregation of usage data can help us find some of these answers and perhaps discover new questions.

Beyond that, I’d love to understand why people are using MetaMask the way they’re using it. Why are people dropping off where they’re dropping? To dig deeper in the ‘why’ behind observed behavior, we will require some follow-up user research.

Christian: At the highest level, I’d like to get a clearer picture of the landscape of who is using our product. There’s no question that MetaMask was built by developers for developers and early adopters. But it’s in the last year we saw a lot of non-technical users download MetaMask .

While our developer community is very active requesting new features, we don’t want to exclude anybody. This includes newbies to the space. Therefore, we need tools like analytics and other data collecting methods, such as user testing, to understand this other group of users and cater the product to a wider audience.

Kevin: Omna, you mentioned that some of these metrics will be a stepping stone into further research questions. Can you tell me how MetaMetrics fits into your strategy of designing and researching?

Omna: With MetaMetrics, we will be able to measure certain success metrics over time. This can help us track if our design decisions are improving the overall product experience.

We are constantly doing qualitative research to test or discover new features and improvements. Combining quantitative insights from MetaMetrics with our qualitative research could enrich our research outcomes. Ultimately, all of this will help us find where and how we can reduce friction & improve experience for users.

We have different types of users on MetaMask. With MetaMetrics, we could get insight into behavioral patterns and needs of these different user groups. If needs are similar across users, we can create a one-size design. But if needs differ, which is what we suspect, perhaps we could add adaptability to our interface. For example, it is possible that a feature is very beneficial for developers while it only adds confusion to a user new to blockchain. Evidence from quantitative data can help us make such design decisions more confidently.

Kevin: Christian, would you like anything add anything?

Christian: This data is going to help us make better informed design decisions moving forward. It’d be really interesting to see where we get to cut some of the fat in our product. We have a tendency to add new features without really understanding what immediate impact it will to the user experience. So with analytics, my hope is that we can take a moment and pause and try to improve the current feature sets we’re currently offering. Curating a product is hard if we don’t take a step back to understand how people are really using our product, we could end up building a bunch none-relevant features that can have a negative impact to the user’s experience. You never really design a product once. It’s a living breathing thing that is always evolving and it’s only going to help us make better decisions moving forward.

Kevin: What are some of the most pressing pain points of our ecosystem/product that you hope that this will address?

Christian: The reality is that today, MetaMask is only one piece of the experience when using this new tech. So we always need to keep an eye on the big picture and in doing so, we can identify and try to address these problems with better UX. At least that’s my goal as a designer. Once you start digging deeper though, you’ll find that there are a lot of technical limitations that we have design around. And many times, there’s only so much designing we can do to help improve or help solve a problem. To speak about our product however, the ones that are constantly coming up are seed phrases management, getting ETH into MetaMask, explaining what gas fees are and how to properly set them. We’ve also done a lot of design work to improve transactions but I believe there’s more we can improve here.

Omna: In general, I see the need of simplifying things in our ecosystem. ‘No jargon’ is a commonly known Blockchain Design principle, but it is difficult to apply beyond a certain extent. We alone cannot solve it for the web3 ecosystem, but we could simplify how people interface with web3 applications.

One design challenge is in deciding, “should we engage users with this technical concept or should we abstract it away?” Showing some of the blockchain-specific complexity could help users understand concepts like why transactions take long and why a transaction failed. Meanwhile, elements like our ‘default gas options’ aim to simplify such concepts for users who are new to blockchain.

Another design challenge is to meet both the user needs and project goals. The people who bring passion to this project make it what it is. As more users start interfacing with MetaMask via Dapps, we’d like to help them experience the benefits and potential this technology offers, whether they bring the same passion or are just there to try it out.

We can only control the experience we enable, so this is a difficult responsibility to take. But in general, I see this as a pain-point in the ecosystem.

Kevin: Given that we are going to be an opt-in model for Metametrics, and we’re making that very explicit, will this have a significant impact on the validity of the data being collected?

Omna: There’s evidence that shows more people tend to share their data with opt-out consents, as compared to opt-in consent. This has been misutilized. A lot! We care about users controlling their data. We will be using opt-in consent to ask permission if users want to share their anonymous data. We prefer if few people share their data intentionally over many people sharing data unintentionally.

Christian: Sure, I think we might not get insights from all group types but that’s ok. As long as we are able to see some level of usage metrics, it’s better than not having any data at all. There are usability issues we may be having that we’re not even aware about. This data along with the qualitative research we’re doing will help us find patterns that will further justify making design enhancements to the product.

Kevin: Is there anything else you’d like the audience to know from the designer’s perspective regarding MetaMetrics?

Omna: I just want to stress that we are not going to collect personally identifiable data, and that we will only use the data analysis for research and development. With MetaMetrics, our main goal it to improve the user experience. If you decide to share your data, you’ll be contributing towards an improved product that benefits you and others in the ecosystem.

Christian: I think we should definitely let these insights inform not just ourselves internally but others in the ecosystem. Since we work at ConsenSys where there’s a bunch of folks are really interested in this stuff, it could shine a lot of insight into their user base as well. I think we can we can definitely open source some of this data so that we can contribute to the ecosystem at large. Create more of an impact there.

Omna: Another point. We have heard requests from other spokes of ConsenSys who interface or want to interface with users who are similar to that of MetaMask. This could be an opportunity for us to share some insights with the community, keep the open source spirit, and help other builders in the space understand how people use web3 applications like MetaMask.