How Etsy Gets Made
Like many of you, we enjoy making things: trying out new ideas, building prototypes, and fine-tuning our work as we go. Etsy is constantly looking for opportunities to improve our products and services, and exploring new ways to make things work better for everyone.
Etsy has a big mission: to reimagine commerce in ways that build a more fulfilling and lasting world. That mission is accomplished in small steps; each change we make to the website or apps helps us build toward that goal. But how do we know if we’re building in the right direction?
To help answer that question, we run ongoing studies and experiments to help us understand how people use Etsy. Members play a critical role in this process, by providing feedback through one-on-one interviews, online surveys, experiments, forums, and prototype groups (where members opt-in to try out new features and provide feedback), to name a few.
Those insights help us look for opportunities to improve members’ experiences. In some cases that might mean enhancing an existing feature, building an entirely new product, or even removing certain things that aren’t working well.
We’d like to give you a better sense of how we use these insights to improve the Etsy experience, and invite you to get involved.
From Big Data to Thick Data
For the past few years, Etsy has placed a big emphasis on “building with science,” designing experiments that help us figure out how to improve certain key features on our site by measuring their performance. By analyzing behavioral patterns over time, we have been able to make several major improvements to the marketplace, such as our revamped listing pages and search results pages.
With over 26 million buyers, 1.7 million sellers, and $2.39 billion worth of annual transactions, Etsy generates a lot of data. Programmers sometimes use the term “big data” to describe this kind of large-scale computational environment. We employ a team of professional data analysts who constantly mine this data for insights about how we can make the marketplace work more effectively for all our members.
In much the same way scientists use experiments to understand whether a new treatment or medicine is having the intended effects on an individual’s health, we use experiments to understand whether a new feature, design, or marketing campaign is having the intended effect on users’ behavior.
What do we mean by the “intended effect”? Our goal is to help buyers discover items they love, and thereby make sellers more successful. So, we might test out a new feature intended to keep buyers on the site for longer periods, or to increase the likelihood that they’ll add items to the cart, or to increase the number of items per purchase.
By introducing a new feature to a random subset of visitors, we can measure how that feature is affecting those metrics using the remaining visitors as a “control” group. We can calculate the difference between those who used the new experience versus existing experience, and that data point becomes one input among others into our decisions on how to proceed. We run these experiments intermittently throughout the year, though we try to avoid running experiments during especially busy times, such as the holiday shopping season.
Experiments help us validate our own assumptions and intuition. They also allow us to identify “dead ends” sooner; rather than spending three months building a new feature only to find it doesn’t have the impact we expect, we can run an experiment early in the process to understand whether we’re on the right path. We consider experiments a way to reduce risk and be more confident that the changes we launch will have the desired effect.
While data helps shape these decisions, it’s only one piece of the puzzle. Recently, we’ve been building on our experience with data-driven experimentation to look for deeper insights about people’s goals, attitudes, and levels of satisfaction with Etsy — things that are difficult to glean from looking at usage patterns alone. We like to call this information “Thick Data” (a term coined by ReD Associates partners Christian Madsbjerg and Mikkel Rasmussen) — deep, rich insights that can help us develop a more well-rounded understanding of our members.
While Big Data gives us a window into what our members are doing, Thick Data helps us get at the elusive “why?” Thick Data comes not from computers but from ongoing real-world conversations with our members. This year, we’ve conducted more than 200 one-on-one interviews with Etsy members — both buyers and sellers — in our on-site usability lab. These sessions help us test our assumptions about why and how members use Etsy, to identify “trouble spots” in the design, and to identify missing features and opportunities for future innovation.
Connecting with the Community
We also regularly send teams out to visit Etsy sellers. Our Seller Visits program provides an opportunity for people from all over the company to get out and visit our sellers, to help us understand the problems they’re facing and develop alternate solutions. Members of the Research Team will also plan ethnographic visits to Etsy sellers in order to observe them in their natural environment. This research has been influential in understanding seller workflows around inventory management and order fulfillment.
Bridging the Data Gap
We also run regular surveys with current and prospective buyers and sellers to track their feelings about Etsy over time, and to understand their opinions about new features or particular parts of our site. For example, in July 2016, we launched a survey about Pattern features by Etsy, a new service that helps Etsy sellers easily create their own e-commerce websites. The survey helped us understand what Pattern features users like best, and which potential new features would be most valuable to sellers who use it.
Surveys help us bridge the gap between Thick Data and Big Data, as they allow us to learn about people’s feelings on a quantitative scale. Surveys often stem from hypotheses and questions derived from in-depth interviews, but the reverse is also true : survey results can help us develop questions that we can best answer through deeper one-on-one conversations.
What next?
“Design is as much a matter of finding problems as it is solving them,” wrote Bryan Lawson in How Designers Think. Those processes often go hand-in-hand, as teams refine their understanding of a particular customer need through ongoing exploration, analysis and experimentation in hopes of minimizing the friction people experience in using a product or service.
Etsy is still a work in progress, and likely will be for a long time to come. As we continue to learn more about our global community of creative entrepreneurs and shoppers, we’ll keep trying out new approaches to understanding their needs. For now, we hope this article provides a useful snapshot of some of the approaches we’ve tried over the past few years.
If you’d like to learn more about our team and the work we do, check out the recent Forrester Research case study on Etsy’s Research team. And if you’re interested in participating in ongoing conversations about the future creative entrepreneurship on Etsy, we encourage you to join the community.
