Motley Fool Premium
A case study in migrating a whole business to a new tech stack
Migrating 18 services into new applications and recreating hundreds of thousands of lines of code to improve maintainability and be more responsive, improve accessibility and provide greater opportunities for A/B testing.
The Motley Fool
1 Product Manager
1 Product Designer / Front End Coder (me)
2 UX Researchers
4 Back-end Coders
Representatives from product, editorial and investing.
Product Design: Illustrator, Photoshop, Omnigraffle, Hotjar, GA
Front-End Coding: HTML, SCSS/CSS, Django/Python, Bootstrap, Git
Product Management: Agile, Google Docs, Trello, Slack, MS Word/Excel
The Motley Fool’s premium services were built on a .NET platform, but after years of testing new individual features built on Python /Django, the tech team was effectively supporting two extensive codebases. The company wanted to consolidate these down to one. That meant migrating 18 services into new applications and recreating hundreds of thousands of lines of code.
In doing so, it was expected to rethink all the redundancy and find a means to streamline the experience for the members and the required code to maintain. The new approach would have the constraints of needing to be responsive and tailored to the growing mobile audience. It would have vastly improve accessibility to catch up to WCAG 2.0 standards. It would be structured in a format to provide greater opportunities for A/B and site-wide testing and learning.
I led the design and development teams through the process of migrating 18 services from the antiquated .NET platform to the modern Django platform. I helped define patterns, make the call on what to migrate as-is and what needed redesigning in the process of porting over. I architected each service and set up the schedule for migration. I also designed an new tier of service that wrapped all the individual services and gave a home to shared tools and elements. I was hands-on in the code structuring Django templates and SCSS design patterns to each cascade properly, greatly reducing the redundancy and amount of code to maintain.
User Research: What Matters to Users
Our UX Research and Product team joints went to work to solve for where we had cases of “that’s how we’ve always done it” to see what may have changed that would open up new opportunities to improve the member experience. The results were gathered using interviews, surveys, mouse tracking and digging into the analytic data.
The picture that the User Research painted was that the feature-creep in the multiple Premium services made them more bloated and harder to understand. The addition of more bells and whistles was welcome but the placement was detrimental, flooding the service with more options and workflows than were needed. The main goal of each service was to provide the investing advice for its collection of stocks. The tools were distracting and had to move.
I worked with our UX team and Product and Tech teams to shape the plan of reducing the walled gardens set up for each individual service, and centralize more service features so features were more predictable findable and not so redundantly integrated into every service
I led the UX team and designed a new workflow and architecture separating service-agnostic tools that was still clearly in the Motley Fool Premium universe but no longer tied to any specific product.
Information Architecture: What Goes Where? What Can Be Shared?
With a new template and branding structure in place, internally dubbed “Premium Commons”, we set out targeted areas of the individual service experience that would be better suited as shared experiences. The following tools made sense to be migrated:
- User Watchlist — The tool available in a handful of services where a member can track the latest news and price on stocks they are interested in could be made into a shared experience.
- User Scorecards — Like the Watchlist, this tool displays stocks the member owns and has more detail beyond news and price to also share cost basis and overall performance return. This too could be made a share experience across all services.
- Stock Screener — The screener provides a means to see all the stocks in a service and filter by categories such as market cap size or sector or dividend yield. This tool could be abstracted to become a centralized service, and if the user joins a 2nd or 3rd service, or upgrades to a new service, this experience should only broaden in scope but not require any relearning on the part of the member.
- Discussion Boards — The boards thrive on having a community interested in discussing a particular company or stock. By having the boards within each service resulted in having redundant boards (like 7 Facebook boards across 7 services) and that only fractures the community and level of discussions to only people in those services. This could be made a shared experience across all services.
- Stock Pages — The fundamentals of a stock, it’s price, it’s chart, it’s P/E ratio do not change from service to service and therefore the pages that provided the key information about each stock could be shared as well.
- Stock Update Articles — Most articles are written from the perspective of the advisor of a service for the members of that service, but there are a handful of articles, such as the notes of the quarterly call, that do not have any subjective advice but rather just retell the details of the news and could use a centralized template to be shared by all services.
I coordinated with the Product team to determine the messaging and communication with members of changes to the service experience and directly engaged with the Tech team to flush out the technical challenges of abstracting these applications to find a suitable shared experience that would be not be more challenging to maintain.
User Interface Design: TopHat and Premium Commons
The rubber met the road at the user interface. All the good broad ideas would need to become very specific and the potential concerns would material or not at this stage. I led the team of designers and developers in meticulously building out the new user interface.
Most of the individual tools needed very little change in interface besides consolidation. With numerous bespoke screeners now merging together that left an untenable number of table columns. The UI team found an approach for offering the larger set of data columns and allowing the member to edit the default table to match the columns that were more important to them.
The biggest challenge for the UI team though was setting up the workflows to make it clear there was this “Premium Commons”, this lobby inside the paid experience but not yet in the service where shared elements like profiles and settings and investing tools would now live.
To make this clearer, I strategized and built with the UI team build a front-page dashboard at /premium/ that included listing all the services and special reports a member could access, as well as linking to the abstracted watchlist, scorecard, boards and screener.
This page was further spec’d out to include a list of the stocks a member was following and whether they were recommended across any service they had access to, as well as any analysis or news of the stocks they followed from any service. This was establishing the new Premium Commons as the dashboard that would be a first familiar stop for members no matter what service they were in, or whether they upgraded or expanded their subscriptions.
Lastly, the new base Premium Commons required a separate navigation from the service navigation. A “tophat” navigation was placed in a black bar atop all pages in all services, that similar to the /premium/ main page, would list out all the services and reports a member could access as well as the shared tools and settings in a predictable established placement, so a user could find their way around from any page in any service.
Content Strategy: Continuity with Advice and Friends
In order for this new structure to be successful, I had to work directly with the Product and Editorial team to determine new workflows and approaches for publishing of content that will make the new approach rich and complete
The most essential item of content that needed to be abstracted was the investing these for the stocks The Motley Fool recommends. On the new shared Stocks page there was links to the individual recommendations from each service but no unifying cohesive investing thesis or latest take. The Investing Analystics and Advisory teams were already teaming up on building out a prototype system for capturing this very information. I worked with both groups to encourage sharing that with the members and integrating it onto the Stock pages on the websites.
Another area that was coming together and needed a new content strategy were the discussion boards. The intent was to migrate all the existing discussions, scattered across services into one central location. This required rethinking where we would encourage members discuss when a stock was a new recommendation (as providing that in the shared channel would not be preferred for the business). New boards were set up in each service for discussing latest recs and other matters specific to that service, while the stock boards became shared by all.
Finally, with the ability to publish content to all members despite the service there required the editors to start to write content that was for all services. There was some in the works already being copy and pasted and published in every service, that was streamlined, but new content pieces like the weekly Foolish Wisdom which shares a broader view on investing came to fruition.
Front-end Development: Code migration
The first step in the migration was to access all the services needing to be ported over and find the common patterns that could be abstracted. So, for example, determining how many of the services had a Community main page with similar structure and messaging and setting to build out one central template, with hooks for necessary variances, to eliminate the redundancies.
This process found about 80% of the core services elements could be shared, leaving the remainder of bespoke features to be ascertained if they still provided value within the service. Some features were retired, some combined and many were set to be migrated over. This allowed for scoping the work and laying out the backlog and series of sprints to get the services converted to Django.
The first service site was the slowest to build. We strategically chose the flagship service to begin, to lead the way. Should anything go amiss or new priorities come about, we were satisfied with the plan that the most important service would be the most forward-facing.
Also new was the use of the front-end framework of Bootstrap 4. This framework decision had its trade-offs but overall make the migration faster and helped meet the responsive and accessible goals as it came engrained in the framework. With the bulk of the first service migrated there was extensive testing for WCAG compliance and across many different devices for mobile optimization. Finally there was a pass through to make sure all the necessary elements were being tagged correctly for tracking with Google Analytics and Hotjar, and that the code could be successfully manipulated in A/B tests with Google Optimize.
With success on all fronts, the project progressed to the next service and the next after. With each new service added there was an opportunity to refine or refactor the shared elements to better work at the larger scale. There was also with each new service the addition of new data points or new UI features or new logic to handle International stocks, or options or mutual funds that were being recommended.
Not everything went according to plan.
- The concept of a wrapper service is complicated to visualize how it would be used and what impact it may have, especially for key stakeholders. Being able to hook into parallel concepts like Amazon’s Prime subscription help, but this became a challenge of needing to over-communicate the concept and benefits and continually try to more get mockups or prototypes made available to have a visual to match the talking points.
- The permissioning for a new wrapper service, Premium Commons, also was unintended trail-blazing. There had not be a product before that was a baseline entitlement for every user in the system. It produces a lot of hiccups for the authentication and commerce tech teams.
- Bootstrap 4 was in many ways a success as it provided so much ease and speed in producing new site templates and layouts. The biggest hurdle with Bootstrap was when looking to make something custom, it was actually harder to have to overcome all the preset Bootstrap expectations first to get back to vanilla styling to be able to incorporate the desired treatment.
There were some great unintended wins
- Once the Premium Commons dashboard was nearing completion the Product team saw how it could disrupt an older dashboard page that was underperforming. Pathways were rerouted and the new page was able to help expidite retiring a system that wasn’t intended in scope but in the longer-term was slated to be decommissioned.
- The high-profile nature of this product at The Motley Fool led to a global campaign inside tech to focus the company to be fully WCAG 2.0 accessibility compliant across all member-facing codebases.
- The universal black-bar TopHat feature was showing such a strong engagement with members, that more teams outside of the Premium product team sought to integrate this feature to scale the continuity of navigating through the Fool universe.
The 18 services were deployed on a new Python / Django tech stack build leveraging the Bootstrap framework and designed to be responsive, accessible and test. This new stock made maintenance and development of the current products drastically more efficient but also providing a model that was leveraged for rapid scaling out of new products. Three recent new products, each required minimal tech and design to set up, brought in (back-of-the-napkin net revenue) more than $25M annually.
Not everything can make it into the release. Here were items for further improving and evolving the Motley Fool Premium effort.
- This effort focused on the Premium products but most members initially interact with the Free website and content and tools. Those live in their own codebase. There is an opportunity now to integrate more of the shared tools and components to the free side of the business.
- The boilerplate product that was used to migrate over the 18 products has since been used to build out 3 more products and has potential for many more. The next step is to be able to stand up quickly and completely a new themed product to meet the opportunities the business sees.
A few key learnings that I took away from this project
- Adding new features can just as easily make the overall product worse. As stakeholders focus on their individual metric and tech teams implement the current stories, it’s easy to overlook the holistic view of how this new feature may actually be making the entire product more bloated or confusing. I took the task of always keeping a watchful eye on the state of the overall product as we were adding, moving or modifying features.
- It’s extremely hard to visualize something you’ve never imagined was possible. One of the biggest challenges with this effort was to get support for the Premium Commons architecture when it was so new and radical from any effort done previously. There was a constant push to morph the execution into something that would have been familiar but utterly ineffective. The best solution I found to overcome this issue was visuals, early, often and as high-fidelity as possible for that stage of the project to get stakeholders aligned on how this will look, feel and impact members and the business.
- Collaboration is key. Engaging early and often with the representatives of Product, Editorial, Marketing, Investing and DevOps was required communication to get everyone on the same page to produce a new infrastructure and architecture..