The Architecture Powering the Wattpad Home Experience

The homepage serves as the primary entry point for users into the many types of content and experiences that Wattpad has to offer. As the platform has matured, we have introduced several new types of content and classes of stories. This in turn has created a need for the homepage to evolve from a simple recommender system to one that must carefully balance content discovery, merchandising, and marketing objectives. This post continues from our previous discussion on Server-Driven UI and will discuss the most recent evolution of our homepage discovery system to one that is able to balance multiple objectives in order to serve content that is most relevant to users.

A Historical Perspective

The homepage at Wattpad has long been considered the primary means to surface our passive recommender systems to users. In its previous iteration, this system was designed with a highly modular architecture that would allow developers and data scientists to integrate new algorithms, or “sources”, into the experience as distinct rows of stories.

An earlier implementation of our homepage recommendation architecture.

This design suffered from several shortcomings:

• The introduction of each new row required changes to be made to all clients of the API, thus significantly slowing down rollout and gating changes behind app updates. A solution to this problem was discussed in our previous post.
• The order of rows, and in turn algorithms, was hardcoded. As a result, the ideal placement for a new algorithm was not well understood. On one hand, the exposure of an algorithm was heavily dependent on its distance from the top of the page and adding it too far below would reduce the learnings gained from its performance. On the other hand, by virtue of being a distinct row, adding a new algorithm to the top of home would be extremely obvious to end users and was observed to artificially increase measured performance due to a novelty effect. As a result, measuring the actual performance and ideal positioning of a new row (which would apply to all users) required several weeks of measurements.
• Since the homepage was directly surfacing algorithms as individual rows, it did not need to keep track of their actual content. As a result, users could often notice extensive duplication when multiple recommender algorithms would converge on the same stories for a given user.
• Finally, in the years since this architecture’s creation, Wattpad had evolved from a product solely focusing on free stories to one that had introduced paid stories on platform as well as extensive off-platform offerings through Wattpad Studios. As a result, there was now a frequent need to surface more diverse types of content to users (e.g. a movie or a physical book). The changes required to enable this often required significant overhead and workarounds in order to work within the framework of a system that had been primarily designed to serve recommender algorithms directly to users.

Diving Deeper Into Ensemble Recommendations

One approach to solving the ordering and awareness problems we just described could be to consolidate our many individual recommender systems into a single monolithic model or architecture. If a single model was responsible for the entire flow of generating recommendations as well as their layout, it would be able to exert complete control over the results being shown to the end user. This would in theory allow the model to not only reduce or eliminate any repetition of content, but also determine the ideal positioning for each story being shown. This approach has been proven out on other platforms, notably YouTube, whose candidate generation and ranking models when put together can be thought of as a singular architecture that drives their homepage recommendations (this approach was described in Deep Neural Networks for YouTube Recommendations). A key limitation of a singular model however is that it would have required us to discard years’ worth of learning and optimizations on existing models and start from scratch. This would have introduced significant risk as there would be no guaranteed baseline of performance for a system that’s driving a significant amount of content discovery on the platform.

The recommendation system architecture described in Deep Neural Networks for YouTube Recommendations.

In order to reduce risk and allow for iterative development, we set our sights on building an ensemble architecture that would allow us to consolidate the recommendations from our many individual algorithms into a unified experience. Here, each individual recommender algorithm could be thought of as providing a “bucket” of recommendations for a given user. The recommendations from all of these buckets would then be consolidated into a centralized “pool” of content that would serve as the finite set from which a user’s final recommendations would be pulled. This consolidation step allows the system to become aware of all the content that it will be recommending to a given user.

A ranked “pool” of final candidates is generated from the candidates of all algorithms.

Once this content has been deduplicated, it is fed into a reranking step that can use story features and user interactions to determine the best order for these stories in the final result. Here, we sought to implement a multi-objective ranking algorithm that would balance engagement, monetization, diversity, and fairness objectives when sorting the pool of stories. Since this step is also able to observe which algorithm(s) recommended each of the stories in the pool, it is able to provide a fair chance to newer algorithms through an explore-exploit mechanism that allows them to gain exposure without requiring hard coded positions. The rewards for each algorithm are computed by tracking the users’ engagement with their recommendations. Over time, this allows the ranking algorithm to determine a unique ranking for each user based on the stories they interact with most, and the algorithms that recommended them.

The final step in this process constructs rows of recommendations from the sorted pool of candidates, with the underlying hypothesis being that users don’t necessarily care about algorithms serving as the demarcations for individual rows. More specifically, we posit that we may be able to create logical groupings of stories from the pool along lines that can be reasoned about more simply than the intricacies of each algorithm’s decision making process. We therefore use a combination of rows with specific selection criteria as well as more generic groupings in order to reorganize the sorted list of stories into a two-dimensional row structure. The ordering of the pool from the previous step is then used to determine the final ordering of rows (i.e. a row containing more stories from the beginning of the pool will rank higher than one with stories from the bottom).

The final rows of recommendations are built from the ranked pool of candidates.

Put together, this collection of steps allows us to move in the direction of a consolidated home recommender system without starting from scratch. While this system has allowed us to address the issue of duplicate recommendations and reduce some barriers in experimenting with new algorithms, there are several areas that require ongoing iteration:

• The multi-objective ranking algorithm is an important area of focus as we try to balance traditional engagement metrics with others that aim to reduce the few-get-richer effect in order to provide a field where any (especially new) writer’s stories can be surfaced to users.
• We would like to build out a more robust system that can decide the final rows of content for a user when fed a pool of stories. We assume that a system that tracks users’ engagement with specific types of rows will allow us to better understand how best to display the list of recommended stories for each individual user. There are also important runtime considerations here since rows are required to understand the relationships between stories in a pool and pull from the shared resource without allowing duplicates.
• Finally, the two-dimensional result produced by our current system assumes that stories ranking near the top of the page have a greater chance at exposure. However, smaller devices (e.g. mobile) that cannot show all stories in a row at once may break that assumption. For example, the 8th story in the first row may have a lower chance at being seen relative to the 2nd story in the 2nd row since the former would require a horizontal scroll through the carousel. The impact of this finding on the ranking algorithm is not yet well understood.

Serving Multiple Objectives

The improvements to the recommender system described thus far would themselves have encompassed a “new home project” in our traditional way of thinking about the homepage. However, as mentioned earlier, Wattpad had since evolved to a point which necessitated the balancing of multiple objectives and types of content on this page. Therefore, we created a system that would, in parallel to the recommender system above, pull content in order to satisfy these additional requirements. These paths each make use of careful optimization and caching to ensure that their response times remain acceptable and that any given path cannot prevent the service from serving a response. The content for each of the four streams is then fed into a “builder” that can organize it into the final user-facing experience.

A holistic view of the parallel paths that produce the final homepage.

Conclusion

The discussed reimagining of our architecture has allowed the homepage to gain more awareness and control over the content that it serves to the user. Since its introduction this new experience has become the primary means of driving passive content discovery, monetization, and marketing objectives across the company.

However, this approach does introduce additional complexity and makes it difficult to predict the experience for users with different locales, languages, and interactions. A system of automated monitoring of all possible combinations is therefore essential to ensure that we do not break the experience for any subset of users. This is a good example of an area in which we require ongoing development to ensure that the home service meets set Service Level Objectives and does not fail in unseen ways.

I’d like to recognize the entire Content Discovery team at Wattpad for their work on rebuilding and continuing to iterate on our home experience.