Driving Retention and Revenue

Three ideas, multiple A/B experiments and one winner

WeddingWire is the leading global marketplace connecting couples with vendors. As a rapidly growing business from its start in 2007, WeddingWire is constantly improving its online experience to delight consumers.

This post is focussed on improving the “Inspiration” feature on WeddingWire which is a collection of ideas, tips and articles about all things weddings.

Background

  • Articles within the Inspiration collection attract large number of visitors. The assumption is that most visitors find these articles on Social Media (Facebook/Twitter) and through organic search on Google
  • The “Real Weddings” photo galleries list vendors which contribute to a successful wedding event. As a result, couples are inspired to open the vendor listing page(s). These galleries are promoted on the Homepage and are the only “inspiration” on the mobile app and webpage.

By adding new features to the “Inspiration” feature, WeddingWire could improve Retention and Revenue. What could these features be? Let’s explore a few ways of how this can be possible — in the form of experiments.

We are going to focus on three ideas in terms of experiments:

  1. Retention — Inspiration Boards, like Pinterest but on WeddingWire
  2. Retention — Search / Wizard, like the existing Color Palette Generator
  3. Revenue — Personalized Recommendations, to show vendor offerings

Experiment 1 — Inspiration Boards

Objective

Just like Pinterest, provide consumers an option to save inspirations. This feature is going to focus on “Inspiration” in the Desktop website.

Data assumption: Save to Pinterest is extremely popular amongst visitors

Hypothesis

Allowing visitors to save, organize and share inspirations will increase time spent on the platform (instead of using Pinterest)

Requirements

When a user clicks on the “Inspiration” menu in the top navigation bar, they are taken to Wedding Photos page. On this page, there are multiple categories which lead to their individual pages. For example, Jewelry. On this page — if a user clicks on a certain piece of Jewelry, they are taken to a page with a picture of just the one Jewelry. The user now has the ability to save this to Pinterest — as a link is provided directly on the image of the Jewelry.

Instead of saving to Pinterest, allow user to save on WeddingWire using a feature called “Inspiration Boards” or “Idea Boards”.

Technical Details:

  1. Show “Save Inspiration” button / icon on the Inspiration image (could be on the “Real Wedding” galleries too)
  2. Re-use existing codebase which is currently used to save vendors. This will reduce time taken for engineering
  3. Allow user to view all the saved inspirations (which can be Cakes, Dresses, Jewelry, etc.). This can be a quick link in the top nav bar.
  4. Allow user to delete any / all saved inspirations. This can be a single delete initially.
  5. Allow user to share the “Inspiration Board” / “Idea Board” page with family, friends, fiancee etc.
  6. This feature should be associate with their existing account

Control Group: No Changes
Group A: Show only “Save to Inspiration Board”
Group B: Show both “Save to Inspiration Board” and “Save to Pinterest”

Test Allocation (create user buckets using following allocations)

Inactive Group — 70%
Control Group — 10%
Group A — 10%
Group B — 10%

Run the test for 1–2 Weeks based on traffic to reach statistical significance

Metrics

We should track the following metrics:

  1. Number of “Inspirations” saved
  2. Number of impressions of the “Save Inspirations” button
  3. Conversion rate (can be calculated by combining 1 & 2)
  4. Conversion rate of the existing Pinterest button (for comparison)
  5. Total number of visits (unique & existing) to the “Inspiration Board” / “Idea Board”
  6. Total number of shares of the “Inspiration Board” / “Idea Board”

Success Metric — Total time spent on the website by users who visit Inspiration pages

The metrics should be compared to existing Pinterest conversions along with traffic which comes from Pinterest.

Components

  1. Design — Work with the UX team to design an icon for the “Inspiration Board”. Also, figure out the flows and design to save inspiration, view “inspiration board” and delete “inspiration board’
  2. Engineering — Work out a solution which takes minimal time for implementation re-using existing code as much as possible
  3. Marketing (and other stake holders) — inform them about the test and keep them in loop about performance of the test

Experiment 2 — Search Wizard

Objective

The current Inspiration interface is based on a “browsing” only experience. It takes multiple hit & trials to find what a consumer is looking for. Finding the most suitable choices should be made simpler.

Hypothesis

Providing users with a walkthrough, wizard type interface to find their inspiration will increase conversion rate of displayed content.

Requirements

WeddingWire already has a wizard type tool called the Color Palette Generator which takes user input in a graphical form and provides best-guess color palettes.

Using a similar approach, a wizard which takes user input like wedding moods, time of year, number of guests expected, type of food, etc could be used to display matched inspirations.

Technical Specs:

  1. A tool similar to Color Palette Generator, so that code can be reused as much as possible
  2. Instead of using the same options, work with content experts to create new interactive questions with easily selectable options
  3. Ask 4–5 questions with multiple choice answers based on question logic. For example, if a user chooses A, A show option A1, B1, C1 and D1; if a user chooses A, B show option A2, B2, C2 and D2; and so on
  4. Based on user responses, show list of inspirations. The list of inspirations could be selected from “Real Weddings” or “Wedding Photos”

Control Group: No Changes
Group A: Show inspirations from “Real Wedding”
Group B: Show inspirations from “Wedding Photos”

Test Allocation (create user buckets using following allocations)

Inactive Group — 70%
Control Group — 10%
Group A — 10%
Group B — 10%

Run the test for 1–2 Weeks based on traffic to reach statistical significance.

Metrics

We should track the following metrics:

  1. Number of users who started the wizard
  2. Number of users who completed the wizard
  3. Number of results displayed after wizard completion
  4. Number of users who clicked on the displayed results

Success Metric — Higher click through rates of the Inspiration articles compared to existing click through rates without wizard

Components

  1. Content — Work with content specialists to come up with list of 10–15 most important questions which can be related to existing content
  2. Design — Work with designers to craft icons / fun user interface
  3. Engineering — Work with engineers to figure out how to effectively select the best matched Inspiration content

Other Considerations

  1. Just like Color Palette Generator, display a banner advertisement for Inspiration Generator on various pages
  2. Launch first on Mobile if possible, as the Mobile platform could be receiving maximum visitors. This would allow faster hypothesis validation

Experiment 3— Personalized Recommendations

Objective

Go to Inspiration and click on Ideas. The page which opens up displays a list of articles, for example:

In such articles, users can be displayed wedding items with a direct link to the vendor. The primary objective is to increase site revenue but at the same time not compromising the user experience.

Assumption: WeddingWire earns revenue when a booking is made

Hypothesis

Displaying item/vendor combination on the Ideas >> Content pages will increase number of bookings.

Requirements

On the recommendations page display a sidebar containing multiple wedding inventories (4–5) linking to the vendors page. The items displayed could be personalized based on personal browsing history, closest-matching-user browsing history, etc. The algorithm could also include affinity to the article content.

Control Group: No Changes
Group A:
Algorithm based on personal browsing history
Group B:
Algorithm based on closest-matching-user browsing history

Test Allocation (create user buckets using following allocations)

Inactive Group — 70%
Control Group — 10%
Group A — 10%
Group B — 10%

Run the test for 1–2 Weeks based on traffic to reach statistical significance.

Metrics

  1. Number of inventory impressions
  2. Number of clicks on the inventory
  3. Number of bookings originated by the visitors from recommended inventory

Success Metric: Conversion rate of the recommended inventory leading to increase in overall vendor bookings

Components

  1. Design — Work with the UX team to design the sidebar
  2. Engineering — Work with the engineering team to design most effective algorithms with time being the biggest constraint. The experiment should be easy to design & release

Summary

In comparison to the three experiments above, I would recommend “Inspiration Boards” as it has the potential to increase repeat user engagement by the highest amount. Also, the engineering overhead of this experiment is lowest due to existing reusable code.

Overall, the goal is to try an experiment, validate the idea and iterate to improve performance of the feature. If the experiment fails, we can learn and work on a different solution to drive similar goals.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.