Product Improvement: Browserstack

Project Summary

The project aims to improve the adoption of Browserstack’s feature to file bugs directly into JIRA for their Live products including App Live.

What is Browserstack?

Browserstack is a company that provides a strong product suite for testers to test for various use cases. Live and App Live are their products for manual testing of web and mobile apps respectively.

As Live is Browserstack’s primary offering, improving the adoption of filing bugs into JIRA becomes very important to improve customer stickiness. This can be done by providing features/solutions that make Browserstack a one-stop-shop for end-to-end testing of products.
In this document we will go through the success metric, primary customers and their needs, how they solve the problem as of now, how competitors solve these problems, and how can Browserstack solve them.

Success Metric

Adoption can be measured by two primary metrics:

1. Breadth Metric: M-o-M % of active users creating at least one bug on JIRA
2. Depth Metric: Avg. No. of JIRA bugs filed per user per month

Definition of active users: Users who have run at least 1 session on either of the Live products in the last month.

Customers

The primary customers for the live product’s filing JIRA bug feature are:

1. Manual Testers in tech companies, testing features before product release
2. Developers (Mobile and Web app) testing feature before release
3. QA teams taking care of end to end functionality of the app before the production release

Customer Needs

All the customer personas defined above have similar needs, which are as follows:

1. Need to track bugs on JIRA end to end from creation to completion from testing platform
2. Need to remind the assignees to provide ETAs and actively work on issues from the testing platform
3. Need to track progress on the JIRA issues created from the testing platform
4. Need to record and attach videos of bugs on JIRA from the testing platform
5. Need to fill org mandated fields in JIRA straight from the testing platform

As Live and App Live are aimed at providing a strong ecosystem to anyone aiming to do manual testing the above needs become pivotal to providing a great UX.

Current Customer Journey

In order to solve for above needs, the user has to go through multiple steps as follows:

1. Open Live/App Live
2. Follow manual tests
3. If bug/issue is found use the Report a bug feature
4. Browserstack automatically takes a screenshot, this can be edited with marker, crop etc.
5. Integrate JIRA via one out of 2 methods
6. Raise a bug, or update an existing bug
7. Add project, description, and other details which include assigning assignee labels incl. Other parameters
8. In case the company has other parameters mandated but not available in BrowserStack, the user will have to go to JIRA to create the issue
9. In case the user wants to upload/attach a video to explain the bug, the same has to be done by first recording the session on Live and then uploading it separately going to the issue on JIRA
10. The bugs created by users have to be tracked regularly on JIRA exiting the BrowserStack platform.
11. In order to remind the assignees about the assigned bug, the user has to track the same on JIRA and then separate messages/emails/calls.

It is quite evident that the user journey of tracking bugs end to end on BrowserStack is broken; this opens up opportunities for competitors to provide a better user journey and get the customer.

This also reduces user stickiness as the Live ecosystem is not strong enough to keep the customer the platform for all their needs w.r.t. exploring and tracking issues found while manual testing.

Competitor Analysis

I analyzed 3 major competitors of BrowserStack i.e. Lambda Test, Sauce Labs, and Perfecto. Following is a summary of the same

Based on the above analysis, it was clearly evident that when it comes to providing a great ecosystem to create and track bugs on JIRA, Perfecto solved almost all the essential needs. Following are some learnings from the competitor analysis:

1. Browserstack provides the best login integration with JIRA by allowing users to log in to JIRA from the platform and integrate automatically. Others still use tokens to integrate
2. Platforms like Perfecto & Sauce labs save session details to re-look & manage bugs, screenshots & videos better helping in creating a complete ecosystem to manage test sessions

Perfecto is very well integrated with JIRA, it provides bug creation UI similar to JIRA, user can add/edit fields they want to fill, they can upload videos, and also track the bugs they created. This brings in ecosystem play by creating a one-stop shop to fulfill all the needs from creation to tracking the bug.

Solutions

The following solutions will try to create a holistic experience for the user to fulfill all their needs and in doing so create customer stickiness.

Attach videos to the JIRA issue

Once the user has recorded a video on BrowserStack, it is stored in the local storage. This feature aims to assist users in creating a JIRA issue with video and related logs from the recording.

The solution requires the following features:

Video recording option on Live — The Live app does not provide a screen recording feature. However, the same is available in App Live. Therefore, we need to create a screen record feature for Live as well. It can sit in the menu on the left as follows:

A logfile from the time duration of the video grab, downloaded along with the video.
An additional option in the create JIRA bug section to attach videos and log file

The proposed user flow is as follows:

1. The user starts video recording
2. Once the recording is finished, a video and a log file in text format is downloaded onto the local system
3. The user can click on report a bug from the menu.
4. This opens the option to report in JIRA. in the menu to create JIRA issue, users can attach videos, additional screenshots, and log files from the local system
5. Post this the user can click on Create/Update JIRA issue

The Bug Tracker

This solution is designed to create an end-to-end ecosystem to create and track bugs be it on JIRA or other platforms. The core of this solution is to create a central repository to track issues/bugs raised from different platforms on Browserstack starting from Live and App Live.

The feature will sit in the top menu bar as follows:

The Bug Tracker will be a completely new section and will consist of the following sub-solutions:

Creating Bugs
With the bug tracker coming in, there will be two ways to create bugs across apps:

1. From the Browserstack apps i.e. Live and App Live
2. From the bug tracker

Managing Bugs
Opening the Bug tracker section will have a list of all bugs filed from the BrowserStack ecosystem incl. Live and App Live. This section will have the following layout:

Here, the user can:
Create new bugs — By clicking on Report on JIRA. This opens up a new dialogue box with all the relevant details that can be added to the JIRA issue just like on the JIRA app as follows:

This would also solve the need for users to fill organization-specific mandatory fields available in JIRA.

Edit/Manage existing issues: This can be done by clicking on existing rows in the table which opens up a dialogue similar to the one explained above just with the existing details of the issue pre-filled.

The following are auxiliary features in the Bug tracker feature to enhance the UX:

Sending issue reminders

Another important need of our users is to remind the assignees about the bug and track the status so that the issues are fixed in time and there is some level of ownership/monitoring on the assignee.

1. In order to solve this, we need to create an additional button for each row, called Reminder.
2. Clicking on Remind gives two options:

Slack -
Clicking on this opens up a new message on slack
The contents are pre-filled with the Issue number, type, and description
The message reads like this: “Hope the issue is on track”
The user can add users to the recipients

Email
Clicking on Email opens up the default email app
The contents are pre-filled
The subject is: “Reminder: <Issue Description>
The user can add recipients of the mail

Edit/Add all JIRA parameters

Creating a JIRA bug/issue straight from the Live/App Live does not have all the parameters. As of now using the bug tracker, all the fields reportable on JIRA are available, and the bug reporter on Live/App Live can also be updated.
This would mean adding all the fields like Design Reviewer, Demo Date, etc. to the “More Fields” button. This would allow the user to fill all the mandatory fields as per their organization required to create the issue.

Metrics

1. M-o-M no. of bugs created per user
2. M-o-M % change in the bugs created
3. M-o-M no. of reminders sent per user
4. M-o-M % of bugs/issues resolved per user
5. M-o-M % active users of the bug tracker (opening bug tracker once in the month)
6. M-o-M no. of bugs created with attachments
7. M-o-M % bugs created with attachments
8. M-o-M % of users integrating JIRA to Browserstack
9. M-o-M % change in users integrating JIRA to Browserstack