Attack tough challenges with dynamic planning
During my time at SEEK Asia, we had a lot of interesting challenges but one I will share today was specific to what we needed to achieve to improve the mobile app experiences for our JobsDB and Jobstreet brands. JobsDB and Jobstreet are job boards which had competed against each other in various South East Asian markets for two decades. These two brands came together under one roof when they were acquired by SEEK and these two companies formed the foundation of SEEK Asia, SEEK’s South East Asian job board business. Almost a year later after the acquisition of Jobstreet, the latter of the two acquisitions, I joined SEEK Asia as CTO.
JobsDB and Jobstreet, which were expending a lot of financial resources battling each other, were at risk of being disrupted by new market entrants in South East Asia such as Facebook, Indeed and LinkedIn as well as some newer, disruptive local market entrants. The opportunity joining together presented was to switch from using those war chests against each other to instead pool those financial resources together and combat the new market entrants.
Mobile apps battlefront
Drilling into the mobile app space which was a key battlefront where there was a big risk of losing market share overall. Both brands had apps which for both businesses had started life built by outsourced development shops and then later in-housed and iterated from there. The foundation of these apps as a result was poor. Most of the feature-set was driven by years of whims with little to no measurement of what was actually resonating or being used. These apps rated very poorly on the app stores.
Adding to the complication there were separate codebases for iOS and Android as well so that meant 4 separate small teams each devoted to delivering a single brand and OS combination. The install base was huge — millions of users across South East Asia for each of the apps we were managing but the ratings and for all of them the user experience was very poor. To migrate millions of users from one user experience to another without significant backwards steps in terms of our position in the market is a daunting task. But the poor experience we were delivering was clearly a key risk for being disrupted by newer more focused apps that offered a better user experience that had already been proven in other competitive geographies. The multiple brands and OS combinations also diffused our efforts, reducing the velocity of any potential improvement impact.
We measured ourselves as a business in terms of our market share of people searching for jobs with our services and applying for jobs using our services. Assessing the apps and the codebases that supported them we knew the effort to bring ourselves to a position where we could compete against global players would be longer than building from scratch. And we hypothesised that a lot of the functionality our bloated apps provided was not actually serving their intended needs well.
As part of making the case to embark on a significant change, we identified what we believed success would look like. We needed to maintain or improve our market share of search and job applications. We identified key elements of achieving this was to maintain or grow our significant install base of users. We experimented with and settled on a cross-platform development approach as our testing showed we could deliver a user experience which was competitive with other new players in the market with only about a 20% overhead for each additional Operating System to support. We also elected to build apps we could theme easily to build once and support the different brands. To a similar end, we also adopted an approach to support what initially would be different backends (systems which were being unified on a different timeline) but would allow for consolidation in the future.
What good mobile apps needed to achieve
That in turn identified further goals that were important to achieving our high-level goals:
- We needed to successfully migrate users of the old apps to the new apps and this informed our MVP and the alpha and beta iterations that followed as we slowly released it to larger and larger groups of users and validated the apps were achieving our other goals.
- Provide a great user experience that was efficient for our users (i.e. we needed to be careful that the volume of screens viewed or actions taken could be good usage or could be indicative of an inefficient experience and needed to be handled with more sophistication) and also lead to productive reuse.
- Having high ratings for the apps on the app stores would be important to improve the prominence of the apps in those stores which would translate to more downloads.
- Reduce/eliminate any crashes (challenging for mobile apps when there’s a rapidly evolving development ecosystem and a longtail of OS versions and devices).
- And so on and so on.
What may be evident to you from this sample above of what was a longer set of goals we had identified as critical to being successful in this challenging endeavour is that these goals are of different resolutions, timescales and concerns. Not all would be relevant when we began but knowing some would be critical in the future did inform some earlier choices we made.
Understanding how goals relate to each other and why that’s helpful
What we could see is there are clear relationships between these goals. For instance, reducing the crash rate correlated with the ratings the apps got. However, another lever for influencing the volume of ratings was how actively your app asked for the user to make a rating and the timing of when to ask etc.
We visually mapped these relationships and analysed these maps to decide where effort could make the biggest impact on our overall goals. One interesting, less apparent area of investment which once identified seemed quite obvious — fast response to app reviews — especially taking action and communicating actions taken in response to constructive reviews could bolster rating scores and indirectly grow downloads. So we ensured multiple members of the team spent significant time reading and responding to reviews and using that feedback to inform the iterative development goals so that issues were always short-lived. Users often updated their lower score reviews with 5-star reviews when their issues were addressed.
For example, one chain of these relationships from most granular and near term to more macro and longer-term might look like this:
Users' concerns are addressed quickly (Respond to reviews within 24 hours, solve issues within a week)
contributes to
Prominence in app stores (Ratings are above 4.5*)
contributes to
Growing install base (Downloads above 10k per week)
contributes to
Most job searches (>45% of all job searches conducted)
contributes to
Most job applications (>30% share of job applications)
This is only one example of the many important chains of relationships which were important parts of us achieving our long-term goals. We maintained a visual map of these relationships which we updated when we learned things. We used our understanding of these relationships to set priorities as we went along. We would share both the big picture as well as specific chains and subsets of the big picture to help communicate WHAT was important and WHY it was important to the various people involved — from developers, designers, product managers, operational teams, marketing teams, leadership and many more who were involved in one way or another.
What was the result?
In under two years, we moved millions (approaching 10 million) of job seekers on to the new experiences across thousands of devices, for multiple brands, for users in 7 countries meeting our search and job application goals, with 99.99% crash-free sessions, 4.6+ star ratings, performant screen rendering and a feature-set we knew was being used successfully, fit for purpose.
The example of this post is an example of just one of the sets of relationships between the goals we used. Visualisation of these relationships helped members of the team and other collaborators get on the same page about what we were trying to achieve at each step of the journey and why we believed it would help us progress towards our goals. Comparing each of these chains allowed us to debate what the next most important investment needed to be based on our current situation and where we were heading. We also would update our diagrams when our hypotheses were invalidated and as such, they also could represent our best understanding at a point in time.
For diagramming goals and relationships we used Miro. First drafts were just brainstorms of what concerns we needed to be aware of and using connectors to draw lines linking between related concerns. More macro concerns gravitated to the top of the page and objectives which might influence achieving the more macro goals were lower on the page. Later drafts were closer to Result Maps as described by Stacey Barr’s PuMP framework. We could then use Horizon Planning to identify a sequence of goals and approximate time windows where it might be important to achieve these outcomes if we wanted to achieve our lofty long-term goals — something I will cover in a future post.
Have you been involved in achieving a very ambitious goal and used any form of dynamic or adaptive planning which helped you attain a positive result? What practices and artefacts did you use? What did you learn? Share your experiences in the comments.
