The Bounties Network is a never ending experiment. It’s a tool to enable humans across the globe to incentivize each other to complete tasks, simple as that! The purpose and requirements of a bounty are up to you and me: they can be set up to serve just about any purpose. As a designer, I’ve never had the privilege of working on a product that I can leverage as a tool to make the product itself better. This post is the first in a series which will focus on bounties for design, how they can benefit the design process, the importance of researching and testing with real users, and how we’ve recently leveraged bounties ourselves to conduct research for the The Bounties Network platform. Meta right?
Design as a Service
On the topic of design, there is a plethora of use cases for bounties that we often consider and have seen on the platform. The ones that come to mind most immediately are design tasks that require work from designers: graphic design, logo design, illustration, etc. These types of design tasks result in discrete deliverables that can often follow value-based or project-based pricing models, and typically don’t require long timelines. It’s easy for a bounty issuer or a “client” to create a logo design bounty and have designers submit fulfillments according to the criteria that the issuer has defined. Design bounties in the past have most often been conducted as “competition” style bounties where many fulfillers submit to the bounty but only a subset of them get accepted. There are a lot of innate and complex problems and challenges around competition style design bounties, but I’ll save that discussion for another time as it deserves its own post.
In an attempt to address some of the pitfalls of competition style bounties, a “Private Bounty” feature recently added to the Bounties Network platform allows for bounty issuers to indicate whether or not they wish to require pre-approval of fulfillers for their bounty. When a private bounty is created, bounty fulfillers must apply to the bounty, and not until they are approved by the issuer are they able to submit a fulfillment. In this way, applicants can be vetted by the issuer, and confidence in the applicants’ abilities to meet expectations can be instilled before work begins.
This is a small step in the right direction as we move forward with the intent of creating a community where fulfillers are respected and compensated fairly for their efforts. We’re building systems designed to set expectations for all parties with the goal of fostering a positive, fair and more collaborative experience. This also proved to be extremely useful in vetting usability test participants for our own user research, which I’ll get into shortly.
Bounties in Service of Design
As we continue to ponder the different use-cases and challenges surrounding design-centered bounties, we are also flipping the definition of a “design bounty” on its head. While there is immense potential for designers to earn via bounties, bounties can also function in service of designers to empower us in the context of our own workflows and processes. As designers in the web3 space, and in this case, as anyone with the desire to have product design decisions informed by real, active and potential users, it’s of the utmost importance to be engaging with those outside of your team, company, and problem space to gather actionable insight into how real people might use or are using your product. Neglecting to actively and regularly seek out opportunities to gather feedback and listen to real users is not only naive, but is potentially self sabotage. This is old news to many working in product, but in a problem space as young as blockchain, I can’t believe my eyes and ears when I hear or witness teams going months and even years without proactively engaging in these kinds of exercises. And no, the Google Analytics from your MVP is not adequate user research. While early quantitative metrics can help you spot trends and identify usage patterns, they don’t tell us enough about the humans that use our products; where they get frustrated and stuck, what they care or don’t care about, and what they express might make their lives as end-users easier.
So why do teams so often neglect to engage with users face-to-face? Some of the more common reasons I hear:
- It’s too expensive
- It takes too much time, which we don’t have
- We think we already know what our users need, and should focus on building that
- We have other things on our to-do list that are more important
- It will yield minimal returns
- We don’t have people qualified to do that kind of research or the people we have in our org are too busy
In general, these are all invalid excuses for young scrappy product teams who really have the drive and willingness to get their hands dirty for the sake of making something their users want. The truth is:
- It doesn’t have to be expensive
- It doesn’t have to take much time at all
- There are few things that are more important for your product when seeking product market fit than well structured, executed, and synthesized user research.
- Stakeholders often won’t acknowledge the potential value-add of these kinds of exercises until they see it delivered, or even better, participate themselves.
- You don’t have to be a bonafide user-researcher to conduct user research with high-value returns. If the experts in your org are busy, do it yourself.
User research can be intimidating. You’re putting your product in front of people for the purpose of having them expose holes, express their pain-points, and tell you, the expert, what you can do better. Yes, you are also doing it to validate your ideas and it’s likely that with criticism comes affirmation of your good work. It’s not all sticks and stones, but that’s really what you’re after. If your aim is to conduct your research with unbiased individuals who you don’t know personally or who exist outside of your circle, you’re putting your hard work in front of strangers. This is way more uncomfortable than just building what you “feel” is right, or what your peers with a technical background and the right domain knowledge tell you is the right thing to build.
Blockchain products are already inherently burdened with a high barrier to entry. The best thing we can do for products as Ðapp designers and developers is have real people use the things we create, and watch and listen intently as they do it.
User Research: A Tale of Two Bounties
So what role can platforms like The Bounties Network play in this process? I could go on about the reasons why we should all be testing our products with users early and often, but there’s plenty out there on the subject, and while I will always advocate, I’d rather do this by giving some insight into how we at The Bounties Network have recently leveraged our own product to conduct research, in addition to some general advice for teams looking to adopt a similar process. Perhaps it will eliminate some hesitancy and encourage others in our community to do the same. My goal is to help others come to the realization that these exercises are quick, cheap, fairly painless (unless you’re building entirely the wrong thing, in which case it might not be so painless, but you’ll still be better off in the end), and ultimately, more valuable than most of your team realizes.
1. Plan Ahead
Determine the type of research you want to conduct. For our purposes, we decided to create a survey for our two primary user types: bounty issuers and bounty fulfillers, to pull from a large pool of users and to collect high-level information about who is using the platform, as well as anecdotal insight into our users’ experiences to date. In tandem, we planned to facilitate a series of usability sessions so we could observe users actively engaging with our product as they completed a series of pre-defined tasks, recording insights and testing our assumptions along the way.
a. Survey Bounties: Running a survey bounty requires deciding whose opinions matters the most to you, whether it’s a subset of the users you already have, or a cohort of strangers who you’d like to engage as potential users. It’s okay to survey several of these groups at once, and advantageous to run different bounties and different surveys for each group so you can budget for each accordingly.
There’s plenty of wisdom available on the web around survey best practices and the ways to craft them optimally in order to maximize their impact: ask about the demographics of respondents, have them answer both multiple choice and long form questions which are right for you and your product, and consider the length and format in order to keep respondents engaged.
b. Usability Bounties: Decide what you want to test. Don’t plan on testing the whole product. Choose 1 or 2 high-impact features or flows that you can have a participant perform or move through as you observe. Determine which touch points you will hit as the participant moves from point A to point B. Write down some key insights you’d like to collect or assumptions you’d like to validate through each of these touch points and use them to inform your discussion with the participant throughout the session. Set aside 1 to 1.5 hours for each participant. How many participants should you test with? Steve Krug tends to think 3 is the magic number, after which you’ll start to notice diminishing returns on key insights. For web3 products, this might not hold as true considering the varying levels of experience that users might have with your Ðapp or blockchain in general (this variety is great for research at the ecosystem’s early stage), but to keep things quick and simple, 3 is a good baseline. You can always run more sessions. For our most recent round of usability testing, we decided to test 2 potential paths:
- Create and activate a bounty. Accept or decline a submission to your bounty and leave a review for the fulfiller.
- Submit a fulfillment to a bounty, and receive your payout once your submission is accepted.
2. Budget for Incentives
Incentives are key to these exercises. When a fair incentive is given, you are able to worry less about things like survey length, and can justify being more thorough in their construction. Bounties sweeten the deal, and are the best way to get people to respond to surveys who wouldn’t otherwise care enough to do them. This means hearing from not just the people who love or hate your product, but also the ones who are lukewarm, and with the right changes, could fall in love too.
Running exercises like these is an opportunity to show your users you care about them and value their time and feedback, and that they are more critical to the success of your product than means of revenue. Your product’s user experience extends beyond your interface. The relationships you establish and maintain with your users is just as important as the ease with which they are able to utilize your product for their needs. Show them this by rewarding them fairly for participating in your design and research processes. You’ll reap the benefits of establishing closer relationships, which will likely result in increased user retention and a future willingness for continued participation in similar activities. For surveys which don’t require as much as an investment (10–20min), we’ve typically paid out $5-$10 for each accepted submission. For more time intensive activities like our usability sessions, $30-$50 is a good base. One benefit of using bounties is that you can always alter the payout if you decide to increase the reward should you feel like your bounty isn’t garnering enough attention.
3. #BountyThatShit
Think through the details of your research requirements. Surveys are easy enough, as we have created survey bounties before to great effect, and have seen that a fair incentive goes a long way in motivating individuals to give considered and honest feedback. For our Bounties users, we were able to send out survey links based on users’ activity as issuers or fulfillers. Additionally, users were sent a link to the survey bounty on explorer.bounties.network to fulfill by submitting their public Ethereum address that they also provided at the end of the survey (effectively “signing” their survey with their address in order for us to attach their completed survey to their bounty submission). Over the course of a week, we received a total of 127 survey submissions, resulting in a robust collection of data.
For usability session recruitment, we took advantage of our aforementioned Private Bounty feature with a slightly different approach. Our usability bounty was created with 2 primary purposes in mind:
- It would function to recruit potential usability test participants by having applicants provide some basic info about themselves and their experience as well as their motivations to participate.
- It would also be used during the usability session itself as the bounty that accepted participants would fulfill as one of the session’s primary tasks: Once we selected our participants from the pool of applications, we then used the bounty during the usability test to have them continue on to fulfill the bounty, with the last action of the test being their receipt of the payout from the bounty as gratitude for their feedback and participation. The bounty can be seen here:
With creation of our usability bounty, we made sure to set a few expectations around the things we were looking for and would require in order for participants to be selected:
- We specified that we were looking for non-designer candidates, and that applicants were not expected to have extensive experience with the product or blockchain.
- We set the expectation that sessions would be recorded with the consent of the participant
- We requested that applicants provide contact info in order to coordinate our sessions
To our delight, over the course of just 3 days we received 28 applications from eager applicants from all over the world, with varying levels of blockchain and Ðapp experience. Despite the fact that we would only be testing with 3 participants, this allowed us to instantly build a pool of potential participants that we could leverage for future test sessions. Once we selected our 3 participants, we went back and contacted each declined applicant to let them know that we would love for them to participate in future sessions, and most were more than willing. With a review of each application and a little digging into the online presence of each applicant depending on what they had visible on their public bounty profiles, we were able to be selective and ended up recruiting 3 individuals from 3 different continents, across a spectrum with regards to blockchain experience as well as native english speaking ability. This was extremely beneficial for us to really place some stress on the design of our product as we facilitated each test.
The Concern Towards Bias
One aside that I can’t go without mentioning is a concern that I and other designers and researchers share around recruiting usability test participants using a platform built on the blockchain. As we look forward optimistically towards mass adoption of the technology, it is obviously beneficial for us to not only be testing with users familiar with web3 dependencies and patterns who can provide feedback around how we can better serve them, but also with those with little to no experience. Unbiased insights are key for designing user-friendly experiences for those who are new to the patterns and concepts that are emerging as the ecosystem matures. In this way, leveraging bounties for usability participant recruitment was a risk and an experiment.
What we were pleasantly surprised to find was that a large majority of applicants were very inexperienced, and fell into the category of “Ðapp-curious”. For many of the applicants, this was their first application/submission to a bounty. One of the participants that participated in our usability session didn’t have a desktop wallet set up to use during the session at all, and I was elated to have the opportunity to gather insights while observing a user on-boarding onto a wallet in addition to using the Ðapp itself.
I expect that as familiarity with the technology becomes more widespread it will become increasingly difficult to leverage platforms like Bounties Network to recruit participants with this level of unfamiliarity with the blockchain, but then again, that is a signal of adoption, and if we notice that trend it can only mean good things. At this point, a shift in focus towards refining the experience for current users of a more mature and widely used product is natural and to be expected.
Takeaways
The purpose of this post is not to go in-depth about the specific user experience insights we gleaned during testing, or to walk through each test touch point or survey question, as that would be exhaustive and that insight is mostly relevant to our team as we iterate on the product. Rather, the intent is to highlight the structure and repeatable process that we’ve built at Bounties Network to efficiently and regularly recruit for and conduct user-centered research using the product. Some takeaways from the experience are definitely worth sharing however, and will hopefully encourage others in the community to join us in similar practices, and to leverage The Bounties Network within your own design and research processes:
- People from all over the world are dipping their toes into blockchain and are experimenting with bounties. For this round of research alone, we had a total of 155 individual submissions from 41 countries over the course of our bounties’ week-long lifespan.
- People are not only eager to trade in crypto but to earn it.To build on the previous point, crypto-curious individuals are spread all across the globe and not limited to the west. This insight alone is a strong incentive for us to be building an accessible and fluid bounties experience that can be leveraged by a range of individuals looking to earn crypto by working for it, rather than being limited to acquisition via investment and trading. We are curious and excited to see which communities and niches start to concentrate and flourish on the platform, and which use-cases surface as the most common for earning in this manner.
- People are eager to participate in blockchain experiments that don’t require prerequisite technical knowledge. One piece of feedback that we received repeatedly throughout our survey results and usability sessions was that people wanted more opportunities to participate in non-technical bounties. Platforms like Gitcoin are flourishing in the context of the developer community, but The Bounties Network is eager to meet the wants and needs of individuals who are not necessarily technically inclined, but are still looking to earn and participate in the movement, and we have consistently validated the existence of this desire through projects like Bounties for the Oceans.
- Not everyone recruited via these methods are blockchain fluent, and that’s great. At this stage in the ecosystem’s lifecycle, it’s important for us to be tailoring our early product experiences to users who are unfamiliar with the concepts behind and benefits of blockchain. It’s the right time to be conducting research like this, and we shouldn’t be hesitant to leverage technologies like bounties to incentivize the un-initiated to join us in our effort towards mass adoption.
Looking ahead
There are many other use cases and challenges surrounding bounties for design, both as a service and in service of. We are excited to continue exploring this niche, and are just as eager to continue to share our learnings along the way. Stay on the lookout for more content around design and bounties in the near future! If you have any questions, feedback, or ideas, we’d love to hear from you. Here’s how you can get in touch and get involved with Bounties Network:
- Join our Bounties Slack community
- Follow us on Twitter: Bounties Network, Corwin Harrell — Design Lead/Author
- Like our Facebook page
- Stay tuned for updates
