“Date a Programmer”: Post Mortem

TLDR; the story about the online dating service with a popular idea, virality, analytics, advisers and thousands of users, but was closed due to non-achievement of the target growth rate.

The source code of the project is available on GitHub at https://github.com/emirn/dateprog

About the Project: Online Dating Service “Date a Programmer” for programmers and nonprogrammers who would like to date, learn a programming language, chat with each other. The service functioned from November 2014 till September 2015.

Website Screenshot (for Registered Users), February 2015:

Results (as of 2015 September):

9,415 registered users
2,000+ MAU (monthly active users)
168,000 messages between users
up to 20 couples per month was starting real life relationship (according to the account deactivation survey)
One couple confirmed had married on September 2015 (they told us)
15+ programming courses were created by users (authors)
1,200 times total online dating users completed these courses

Catchy Idea or Stable Business

Catchy ideas do not always mean the business itself will be great too. Passion comes and vanishes, but the business (and its profits and losses) remains. A bright idea makes users expecting high quality too. So, if the implementation “fails” you may get critical feedbacks and frustrated users.

CORRECT APPROACH: The catchy wording of the pitch, understandable to everyone helped to start the project and generate the first 2,500 users with minimum expenses for their involvement, just by creating a single landing page using www.quickmvp.com.

WRONG APPROACH: A bright viral idea is not enough for the decent business. It was necessary to examine the data from market insiders and study interviews with them, as well as market data and forecasts (dating industry grows by single digit number per year, dozens of dating startups starts and fails every month). Before launching the project, we should be better test existing traditional dating services, including paid ones, as well as examine the readiness of the audience to pay.

Table with Emails Collected by the Landing Page

Users First, not Programming

Only after gaining more than two thousand users (leads), we started the programming itself. Upon implementation of the first version of the service, we cleared the email database from non-existent emails, doubled emails and uploaded it to the service. Then we have sent invitations to users asking them to join the service: first, we have sent 50 messages, next day we have sent 100, and on the following days, we have been sending to batches of 200 users. So, this way we distributed the load and simultaneously were fixing rough errors and bugs in the service. Among all leads who provided their emails on the landing page about 60% activated their account on the live service. That’s how we got more than 2 thousands of users on our service, and they started to uploading their photo, filling profiles, etc.

CORRECT APPROACH: We did not start programming until we gained a critical mass of people interested in trying the service. We launched the first version, and then we seeded batches of new users to keep track of key issues and bugs instead of planting the whole bunch of leads at once.

WRONG APPROACH: On the landing page we were asking just for email and the simple description. It was necessary to immediately try to collect data and measure how many of these leads were willing to pay for the service (and for what they would pay). This helps immediately, even roughly, to estimate the free to paid users conversion. It is great having success to reach the target audience, but there is the need to understand what kind of users you will work with, and how much money they have and if they are ready to pay for the service or product like this.

The Most Active Users in MixPanel for 3 Days

Duct Tape

We had no experience in creating and launching the online dating services. The first version made was running slowly; page loading time was about 10–12 seconds. To simultaneously keep 400–500 users, we had to run five servers on Heroku. Only a few months later, we found an error that caused this delay. By correcting it, we accelerated page load time and was able to decrease the number of servers to 2.

RIGHT APPROACH: The primary focus was on the changes and the speed of changes. Users were swearing but continued using the service. One of the users sent us the thank you letter telling that the slow speed helped to move to the offline meeting instead of indefinitely chatting.

WRONG APPROACH: We did not use third-party services to implement core functionality, though later we have discovered that there are even white label services for online dating services that we could use instead of writing the code. We spent resources on reinventing the wheel and what is more important we spent time.

Users’ Feedbacks on Russian Public Forum -lot of errors and issues with the basic functionality

The Two-sided Market

The two-sided market is a chicken and egg problem: what should appear first? But even upon launch, you have to sell to two different audiences. It is necessary to find out which audience is the dominant one, who is the “seller” and who is the “buyer” what is the behavioral model of each type of audience. First, we focused on ladies. Shortly upon launch, we got into the situation when there were thousands of ladies, but the number of males was ten times lesser. We began to receive complaints from women: they logged in, wrote messages to few programmers but got no response. After making publications on few popular forums and Habrahabr (Russian YC News), the situation improved. By February-March 2015, the ratio of 2 audiences had been about 50/50.

CORRECT APPROACH: First we attracted ladies as the core audience. It was easier to gain them first. Male users would be much more complicated to bring without ladies.

WRONG APPROACH: Later on, we tried to satisfy both types of users at once. Programmers wanted to have a cooler terminal like a console, API access, and other geeky things, but ladies asked to have more convenient chatting interface and improve the usability. As a result, we got no x10 improvement even for a single selected type of audience.

Terminal Version of the Website for Programmers:

Advisers

While implementing the project, we got three advisers and one mentor involved. The mentor helped to focus and launch the project. And then consultants assisted in running the project at different stages.

For example, Ilya Osipov, a freemium expert helped to build a systematic approach to gain users’ retention and to write a chain of 11 emails, as well as to implement and test some viral mechanisms. Another adviser in start-ups in the field of online dating helped with the information on the state and direction of the dating market, the viability of various business models and contributed to set the goal of for the next stage of the project (to gain 1000 daily active users). The third expert who is running the dating services himself helped us with insights of the CIS (Russia, Ukraine, etc.) market, user acquisition costs helped us to understand better the audience, dating users’ behavior and characteristics.

CORRECT APPROACH: Experts and mentors are required to start new projects, especially if you do not have experience in this niche market. If an expert has expertise and experience in this particular niche exactly with this audience, even a short chat with this person can significantly influence the project. The question is to the expert interested: you must also share interesting insights and useful information if you have some.

WRONG APPROACH: It would be better to consult with experts first before launching the project. Also, some advice given by experts was ignored but then proved to be the only way to go. For example, while implementing the viral mechanics, we found the mechanics suggested by Ilya Osipov as the most efficient by comparing and testing different virality implementations.

Table with email triggers proposed by Ilya Osipov

Haters Gonna Hate

Immediately upon launch of the service, because of its slow speed and various technical issues, we started getting abusive emails and feedbacks from our users. The ratio of critics and thank was 10:1. We got a lot of aggressive comments with personal attacks. When starting a free project with obvious technical problems you should be ready for angry and aggressive comments and even insults.

We were constantly conducting internal surveys (using a banner inside the site containing the link to the inquiry on Google Forms). We asked: ‘Are you willing to pay for access to the service?’ And we began to get insults asking how we can even dream of asking to pay for this sloppy site. After changing the question to: ‘We are investing a lot of efforts into the service, but the service can not evolve without any support. What would you be willing to pay for?’, some insults drastically decreased.

As our audience included Russian and Ukrainian programmers so were threatened by DoS attacks few times. Somebody was also was trying to make fake accounts (so we had to make a blacklist of email addresses). We also had to add the automated block of an account once it got three or more complaints from other users. A constant number of allegations and insults with the subtle amount of positive feedback demotivated us.

CORRECT APPROACH: We conducted online chatting with the most active users in order to understand what they really like in the service, and what they don’t. Thus, we revealed several serious issues and made improvements. We conducted surveys: for example, several dozens of great tips on how to start a conversation with a stranger were collected from users just for two days.

WRONG APPROACH: We should not be taking seriously all the critics and negative feedbacks, especially from free plan users. Instead of reflecting on emotions, it was important to focus our attention on the growth of specific metrics of the project and pay the attention to the audience that was confident and willing to use the service.

Critical Feedback on Public Support Forum Suggesting that even Sailors are better than programmers

Virality

The primary metric of any online project, especially an online dating service, is the growth rate.

First, being inspired by articles and books about the gamification, I calculated a gaming like model where the user is charged or withdrawn “credits” to/from his personal account for every action (if the user opened a chat, received a message, viewed a profile, etc.). I calculated scenarios of a typical guy and lady and adjusted the general model so that as soon as the user went beyond the typical use case, he/she is asked to invite friends to continue. It looked good in Excel, but during the implementation, we faced the need of the deep code refactoring. Moreover, it was hard to explain even to the programmer how it would work. If we cannot explain it to each other, will we manage to explain it to users?

Table with Gamification Models

So, we created a new model called “battery” that was slowly “charging” while user was on the website, but also “discharging” while he/she was making any actions. According to the model, after 20 minutes of browsing profiles and chatting the battery should “went dead”, and the user had to either wait for a half an hour or charge “the battery” by inviting friends to sign up too. It looked perfect in Excel spreadsheet.

The reference to mobile phones is perfect for an explanation: the battery goes dead if used for some time, so you have to either charge it or get an infinite battery (by purchasing a premium plan).

We launched this model, but Analytics was showing that when the “battery” was dead, and the popup window containing explanation appeared.. in most cases, users just closed the popup. We have tested this model ourselves on the website and got found that it was still too complicated. The virality rate of this model was about 0.16%.

Pop-up Window Asking To Charge The Battery

The last model was a very simple one. As originally suggested by Ilya Osipov, we have created a kind of a progress bar with the number of invited users asking to invite nine users more to signup (Ilya described this mechanism in his article (Russian)). Next to the main menu, users could regularly see the progress bar with the number of already invited friends and the number of friends remaining to invite. Upon nine friends’ registration and confirmation, the original user was getting a VIP account with a highlighted portfolio and the function to upload three more photos. The message to users got simpler (comparing to previous models) and reduced to the short phrase: “Invite 9 friends and get a VIP status.”

With this new simple design, the virality increase by 11 times and now was 1.83%! Later, we tried to test even simpler version of the model with just two friends to invite, and the virality grew even more to 3.29%.

The message that asks to invite two friends and get a VIP status

While doing experiments, it is crucial to select a proper analytics tool. First, we used Google Analytics which is free and familiar to everyone. But it turned out that a part of events was invisible and was not registered correctly. Moreover, we had troubles implementing the grouping of events by users. After a brief search, we found MixPanel service. It allowed us to log events, count number of events and group events linked to user profiles. We then understood why so many startups are using MixPanel. Also, they are offering a free plan (with the limitation on the number of total events registered per month asking to place their service banner on the main page of your website in exchange for the free plan).

CORRECT APPROACH: We measured results of our experiments with the virality, and these results allowed us to find the most efficient model that works best for us. Unlike overwhelmed GA, MixPanel enabled us to execute the convenient tracking of our virality implementation.

WRONG APPROACH: We lost a lot of time because we went from the compound model to the simpler one in the process of experiments with the viral. We should be better going in the opposite direction, from a simple design to complex ones and without trying to reinvent the “wheel” first. We lost time and data while was using Google Analytics and wanted to make it work for us instead of trying more suitable and user-friendly MixPanel.

Events registered in Google Analytics

Event-Based report for New Users in MixPanel:

Conclusions:

• Startup means working in resource-limited conditions. It may seem that there are always a lot of opportunities and marketing channels available. In reality, you need to choose, for example, how to spend your time: whether you want to improve the product or write an article? Do you want to integrate a third party service or spend time and money creating your own?
• Each industry has its little secret which is apparent to insiders but not that clear to others. For example, with an online dating, the “secret” is that people are using free online dating services not to find someone but in most cases just to have fun. It’s the way freemium and free dating services work. We did not understand this little “secret” correctly.
• A viral idea helps to start a project. But apart from a viral idea, we must not forget the economics of the project and its business strategy. A viral idea could bring a user to the website, but the idea is not enough to keep users attached to the service.
• Virality should be simple, clear and logical. In the case of online dating, it ‘s hard to expect that a user will bring her friends to join the website because they could act as her competitors. It is necessary to set up a mechanics to provide mutual benefits for users to keep a long-term presence of their friends on the service. For example, joint/team games, creating or sharing the content with each other.
• Advisers and mentors are the essential part of a startup as they are helping to find the right strategy (or abandon an existing one promptly), direct you in the right direction and as a result save your time and resources in a significant way.
• Before starting a project in an unknown market, it is necessary to study the leading players in the market and put yourself “in the shoes” of their users, understand why users are paying to them. It is a good sign if there are successful competitors in this market especially if they are growing and making profits. And it is dangerous if you are the first one to start this kind of product thinking that you are an inventor.