The quest for product-market fit
A Lean Startup case study on PaintBerri
The past two and a half years I have been working on PaintBerri.com, an online community for artists to draw and create visual stories with other artists. This month my co-founder and I are returning to regular jobs in our respective fields after taking almost a year to focus on PaintBerri full-time.
The following is an overview of how the team made several important customer insights and product pivots through the Lean startup approach: using MVPs, validated learning, and an efficient feedback loop. My hope is that this case study inspires other entrepreneurs to go Lean, and that others will also have the opportunity to learn from the challenges the PaintBerri team faced.
Please note that this case study only hits on key turning points in the product and business — all throughout this story we were running tests on experimental features, marketing campaigns, and more.
From side project to full-time project
PaintBerri began as a side project in mid 2014, led by Katherine Tung, a software engineer, who wanted to create a superior web-based painting program for digital artists and sharpen her engineering skills.
The site was inspired by the communities formed around web-based drawing programs called Oekaki boards, popular with the online art scene in the late 2000’s.
In those early days, I contributed mock-ups and graphic designs when I had time on the weekends.
Once the basic version of the painting program was operational, the team invited artist friends to take it for a spin in a closed beta, and help find bugs.
The closed beta was a huge success, with many people online clamoring for more access codes to share with their friends. Katherine quit her full-time job at Expedia and dove into building PaintBerri into a full-fledged online community with a built-in painting program. In late 2015, open beta was officially announced, and the floodgates opened.
The subsequent rush of over 6,000 artists, the chaos that ensued, and my own jump into the fray, set the stage for the implementation of Lean principles. We were already running Lean in a sense, by bootstrapping the business and running it with minimum expenditures, but we soon learned that it’s never too early to run Lean.
Once I moved back home to Bellevue, Washington after a tearful goodbye to my friends at Udemy in San Francisco, I got to work researching how to run a startup. I devoured stacks of books, several of which I recommended in another post, Books I wish I had read a lot sooner.
This sets the stage for the first phase of the Lean cycle: Build. We had already built a online community centered around a painting program. All the art and comments published on the site were created with this painting program, and we were just beginning to iterate on the product.
The first version
Users could create a profile, draw online, publish their drawing, and comment on posts with mini-drawings. Comment chains would often become improvised stories, called “roleplays” such as this one:
We also had a pretty good picture of who was on PaintBerri — mostly young female amateur digital artists who enjoyed anime, manga, movies, comics, video games, and nerd culture.
Going down the feature rabbit hole
We were signing up many new users each day, but we were also getting a lot of complaints and bug reports centered around the painting program. Naturally, almost everybody presented their request or bug as critical to the experience, which was overwhelming. We added tons of features that directly addressed these concerns, but it felt like no matter what we did, people would never be happy, and that was frustrating.
Soon, we knew this wasn’t going to be a sustainable way to make product decisions. It was also worrisome that all the bug fixes and updates weren’t making any noticeable impact on user engagement.
Challenging core assumptions
After releasing dozens of bug fixes, features, and enhancements to the painting program without making a dent in the deluge of complaints and bugs, we paused development.
What if artists were coming to PaintBerri and staying for reasons other than being able to use a free web-based painting program? Unfathomable!
Setting up a study
To find out, I designed and implemented a research study to understand why people liked and disliked the PaintBerri experience. I read Cindy Alvarez’s excellent book, Lean Customer Development, and designed an interactive survey inspired by the recommendations in the book.
Digging deeper to find the motives
The survey was displayed to logged-in PaintBerri users for several days, and we gathered over 1,300 responses, plenty of data to see clear trends. Thankfully, most people were having a good time on the site!
The breakdown of the responses to “what do you enjoy the most/least about PaintBerri” is where it got interesting. Roleplaying and art sharing were the top positive features of the site, and the painting program was top negative feature.
To get a qualitative angle on this data, I set up interviews with 10 of PaintBerri’s most valuable users who were screened through a series of surveys, then hand-picked. These were artists that were very active, passionate about the site, and consistently generated valuable content — art posts that had a high number of followers, favorites, and comments. We wanted to attract more of these people, so first we wanted to understand why they were on our site.
From these interviews, we learned 3 factors that keep these artists hooked on PaintBerri: a close knit community that was more open to interacting with other artists (as opposed to the sprawling and impersonal communities of DeviantArt and Tumblr), positive feedback and encouragement from other artists on the site, and a place that supported visual storytelling in the form of comment threads (roleplaying).
Deciding the next move
It was pretty clear from the survey results and interviews that socializing with other artists and creating visual stories through roleplaying were our user’s top priorities.
This marked a large shift in our approach to the PaintBerri product, since it was conceived as primarily a web-based painting experience, with the community a secondary aspect. Now we were turning that approach on its head, and we had a lot of catching up to do.
Making the pivot
A few major changes marked the pivot to a PaintBerri focused on bringing artists together to create visual stories. We paused development on the painting program, allowed artists to upload work created in other programs, and began launching MVPs of social features, such as a mini-forum. The changes were announced in a letter to the community that was well received.
Putting out one fire, and turning to face another
Immediately, we noticed a drop in the number of complaints, and many artists that were having compatibility problems with the painting program were especially happy with the art upload option.
But, there were some other aspects of running a sustainable business that we had been ignoring for too long.
Putting the spotlight on monetization
User satisfaction didn’t translate into profits very well. We were relying on advertising for revenue, but despite the increase in users, it wasn’t going to be a long-term source of funds. We were also having difficulty with rotating ads on the site, which didn’t work well since PaintBerri is a single-page app, built in AngularJS.
We’ve run small fundraisers in the past that were successful, so we decided to put on a larger one and use it as a way to test how well our market converted from free to paid users.
Testing if our users would convert to customers
We designed and built a donation campaign with bonus roleplaying features as incentives. All of the bonus features were nice-to-haves, like a little token that appears after the username and more characters per comment. The most expensive donation tier, the Diamond Heart, would get you all the bonus features for $99 plus an exclusive new feature — variable height comment canvases. We put it there as a price anchor, not thinking anybody would actually buy it.
Success, and a not-so-obvious failure
Amazingly, in under two weeks, we made 250% of our original fundraising goal. Dozens of people bought the Diamond Heart, which we thought nobody would touch. We were stunned.
But there was a fatal flaw to our success. Looking into the data, we had managed to convert a high percentage of a minority user segment — the early adopters from the closed beta period. New users, who represented the overwhelming majority of PaintBerri’s audience, were barely converting.
In PaintBerri’s early days as an online painting website, we knew it was catering to an extremely niche audience. Many of the early adopters were on the site for nostalgic reasons, and were thrilled to see one of their favorite online experiences from the past, revived and modernized.
By evolving into a social art roleplaying site, PaintBerri was in the process of alienating many of these early adopters, who valued the focus on improving one’s drawing ability by limiting all artists to publishing work from the built-in painting program. Unfortunately, these were also the people who were more willing to pay for such a product.
Furthermore, there were no business success stories we could find for other niche art and roleplaying sites. Even the relatively well-known ones were run by volunteers and supported through donations. The only exceptions were art sites which featured NSFW/explicit content, and we didn’t want to go there (for obvious reasons).
Knowing when to stop
From the failure of the donation campaign and the lack of other successful (and non-explicit) niche art and roleplaying sites, we had to come to the difficult conclusion that monetization was going to be an incredible challenge. Part of the Lean philosophy is running as efficiently as possible, and that meant putting PaintBerri into stasis to conserve resources which could be later applied towards products with a higher chance of return on investment.
In July 2016, we posted a letter to the PaintBerri Blog to announce our decision.
A potential future for PaintBerri
Even though PaintBerri is in stasis, its not the end of the entire product. We learned that in our initial quest to fulfill the endless list of feature requests, we have created an incredibly powerful browser-based painting program. Where most browser-based painting apps use Flash or Java, PaintBerri’s is built using WebGL. Katherine talks more about the technology here in an interview from the digital art tech blog, SurfaceProArtist.
There are many other possible applications for this technology, and opportunities to find a better product-market fit. If we had run Lean earlier, we would have been able to vet these applications much faster, before we committed so much time, energy, and money into the effort.
Looking back, and looking forward
PaintBerri is still an incredible success in its own right, and it far surpassed my own expectations for what it could become. What began as a fun weekend project became a site that gathered over a million page views a month, attracted rising stars in the digital art world, and helped young artists from around the world build lasting friendships. And we did this all while keeping 100% ownership of the company and keeping our personal savings intact.
To date, over 23,000 artists have created accounts on the site, and have collectively published over 100,000 posts. Many of these artists are incredible superstars, look at their amazing work here: Yawns, Zephy, Goons, Azareal, Sugar, and Belacqua.
I’m incredibly thankful to have the opportunity to be an entrepreneur. It wasn’t easy to leave a well-paying job, and I know many people don’t have such an option. Through this experience, I’ve grown so much as a designer and a person. My eyes have been opened to the fascinating inner-workings of product management, marketing and business, and I’m excited to bring these learnings with me to my next design role (and eventually, my next entrepreneurial adventure).
Much gratitude to PaintBerri’s dedicated mentors and advisors: Dennis Woo, Randy Chung, William Yu, Diana Liu, Ryan Waliany, and Serena Wu.
And many thanks to Pablo Stanley, Robbin Arcega, Scott Rabin, Ibrahim Damlaj, Rachel Lee, my parents, and all the incredible PaintBerri artists for their support.
Let me know if you would like to chat more about my experience :)
If you enjoyed this post, please recommend so that more people can discover it, thank you!
