CMU Capstone Project — Improving Engagement on the 99P Labs Developer Portal

Written by: Bhavya Sachdeva, Emma Wang, Kunaal Khilnani, Tushar Prakash
- Carnegie Mellon University MSPM Capstone Project

Introduction:

Over the last four months, we had the opportunity to collaborate with 99P Labs for our graduate program’s (Master of Science in Product Management) semester-long capstone project. We were delighted to work on a really exciting problem that helped us hone some of the key concepts that we had learned as a part of our curriculum, such as analyzing KPIs, writing user stories, conducting user research, and prototyping. Our problem statement was to increase organic engagement for the developer portal of 99P Labs (https://developer.99plabs.io/home/).

Approach:

We utilized the double-diamond design-thinking methodology taught in the MSPM program to break down the problem statement into phases. At first, we decided to think big to understand the users’ problems through customer research. By conducting several user interviews, we were able to identify key pain points that had a direct impact on our project goals and objectives. In the second phase, we prioritized those pain points to identify potential solutions. Ultimately, our team had proposed seven solutions, out of which we found four of them to directly impact user engagement. Finally, to validate our proposed solutions, we created high-fidelity prototypes to elicit critical feedback from our end users. Throughout the process, we frequently met with the 99P Labs team, who were always happy to share their experience and expertise and provided additional support whenever required.

Phase 1:

User Interviews and Google Analytics

In order to shape a successful strategy to achieve our goal, we needed to discover things we could have never guessed about the users (researchers, developers, etc.) and their behavior on the platform. So we conducted user interviews and leveraged the site’s analytical tools for the same.

We spoke with a few power users of the 99P Labs platform to understand the reasoning behind the low user engagement on the platform. These sessions were highly informative, and they helped remove our internal bias, take a user-centered approach and reduce risk. One user mentioned that the datasets on the platform were extremely helpful but location-specific. He wanted diverse datasets to continue his research. Whenever he found himself stuck during his research project, he searched out in the FAQs or reached out to a co-worker for help. Another user mentioned that 99P Labs allowed her to work on an impactful project where she used datasets to predict where a vehicle would go, and it’s dwell time. However, she mentioned that some tutorials or more documentation around issues already identified by other researchers would have helped her reduce time to value. Moreover, she relied on other platforms to collaborate with or seek help from her teammates. The users also agreed that some indirect incentives or recognition for research would help customers engage more and retain on the platform.

Google Analytics also revealed some valuable insights on what people were searching for on the platform. For example, we found that Data Sets and Documentation pages were not accessed frequently, and the bounce rate of the platform was high. Bounce rate was the most important metric as it validated that the experiences of the power users we interacted with applied to a larger audience.

With this analysis, we were able to map out a typical user journey and were able to bucket the journey steps into four categories: Discovery, Registration, Onboarding, and First Use and Sharing, along with the actions users take and their needs/pain points at each step.

Discovery: We identified that the users only know about the 99P Labs platform because their university has some connection with 99P Labs.

Registration: The portal provides a variety of unique datasets and developer tools, but the potential user has to wait overnight to get access to the platform.

Onboarding and First Use: The interface is interactive, and the staff is very supportive. Users rely heavily on the support documentations and are interested in learning through collaboration.

Sharing: The users do not have a one-stop platform for researching, working on their code, and collaborating.

Phase 2:

After scrutinizing user pain points, we generated eight features and put them into three buckets with specific functionality: a. Increase Engagement; b. Reducing Fiction Points; c. Enhancing Loyalty/Advocacy.

Increase Engagement

1. Feature: Collaborative coding platform
2. Description: Set out a coding platform that enables team collaboration on 99P Labs.
3. Rationale: Users regard 99P Labs as a merely downloading platform. After getting the dataset, they couldn’t find other reasons to stay or return to 99P Labs. Hence, we suggest that 99P Labs provides a collaborative coding platform to allow users to fulfill teamwork onsite.
4. Feature: Internal community for Q&A
5. Description: Build up an internal community for existing users and newcomers to conduct Q&A activities.
6. Rationale: Many existing users reflected that when they had questions about dataset or research, they had to contact 99P Labs or seek help from this website. With an internal community, existing users could share experiences with new users on that platform and help newcomers facilitate the progress of their research.
7. Add more documentation and auxiliary data analysis like EDA
8. Description: Add more auxiliary data analysis to help users quickly visualize the dataset.
9. Rationale: In our interviews, some users said they felt confused when processing the dataset downloaded from 99P Labs. So we suggest that 99P Labs provides more assistance to help users comprehend those datasets more efficiently.

Reducing Friction Points

1. Optimize onboarding process
2. Description: 99P Labs optimize the onboarding process for team-based researchers.
3. Rationale: 99 P Labs utilized an approval-basis policy to allow new users to onboard. We suggest that 99P Labs reduce the waiting time for new users, especially team-based researchers, to start doing their research quickly.
4. Diversify dataset:
5. Description: Add more diversified datasets
6. Rationale: Users are likely to utilize various datasets to support their research, whereas 99P Labs could scaffold them by adding more datasets based on their needs.
7. Enhancing Loyalty/Advocacy:
8. Badging- Indirect Incentives
9. Description: Build a badging or certificate system to allow users to share those incentives on professional social media platforms.
10. Rationale: Users need recognition and exposure for their researches. So that we suggest 99P Labs could design a system that grants users who completed their research a certificate or a badge and enable them to share on social media.
11. Organize contests
12. Description: Set out contests on 99P Labs so that users could attend and compete online.
13. Rationale: 99P Labs own valuable datasets and resources. To increase user engagement and stickiness, we suggest that 99P Labs could enable users to join contests on 99P Labs, which helps 99P Labs gains more user awareness and loyalty.
14. Subscription emails to inform the existing users about updates
15. Description: Enhance the email notification for users
16. Rationale: Many users said they would like to return if 99P Labs updates any new features or datasets. However, they didn’t know how to get such information. Therefore, we suggest that 99P Labs enhance the email notification to inform users about the latest updates.

We invited a few target users from CMU to validate our ideas in the next step. Then, based on the stakeholder impact, Market Impact, and Efforts, we prioritized four out of eight features.

The next steps for this project detail prototyping for these features and running human centered design and testing processes on these features. This will be continued in part 2 of this blog series. Follow 99P Labs on medium for alerts on when this will be released!