7 Sure Ways to Fail at Personalization

Do you really think the world revolves around you?

Umm … not yet, but they’re working on it!

Aah, that feeling when personalization gets it right: a news feed that provides relevant updates, a virtual assistant that learns your interests, an app that changes its workflow based on your repeated usage. It’s a magical feeling. Unfortunately, it is still a rare feeling. The reality is that most products are thinly personalized, poorly personalized, or not personalized at all.

If personalization is the best thing since sliced bread**, why exactly is it notoriously missing from many of the products and services we interact with on a day to day basis? Why is it not imbued into everything from user experience to content generation, making our personal and professional life better? It turns out that getting to personalization at scale is not that easy.

Over the past decade I’ve been working with companies in several verticals — fintech, healthcare, social, advertising, security, communication, infrastructure, etc. — helping clients build products and platforms spanning the personalization spectrum. In my engagements I noticed a handful of misconceptions that are responsible again and again for stalling, obstructing, or derailing sometimes massive efforts to get personalization off the ground. I’m sharing some of these misconceptions here.

(1) Machine learning equals Personalization! Pitfall: believing that machine learning (or more amorphously “AI”) alone is all it takes to do personalization.

Yes, machine learning is the engine that makes personalized experiences possible. Yet it takes a lot more to achieve personalization at scale: product instrumentation, timely data collection, fast data processing and feature computation, secure and flexible infrastructure to scale to hundreds or thousands of models in development and production, automation, security, etc.

To reach personalization at scale, it is quite common to spend your early time and energy on platform, data, and user feedback integration, rather than on machine learning algorithms. However, this reality is rarely reflected in most plans for personalization. Instead, the discussion is centered around ML capabilities: e.g. “so … will we be able to do deep learning?”

One thing is clear: AI will not subjugate humans without a good platform, solid integration, and thorough automation.

(2) Hire 99 data scientists! Pitfall: building a large data science team for personalization too early, well before you can leverage its talent and skill set.

In their desire to personalize products, companies sometimes rush to hire disproportionately large numbers of data scientists early on. This can be quite disastrous if they do not have a solid and scalable data platform — from consistent instrumentation all the way to flexible access to data and computation. In most cases it is a knee jerk reaction in an effort to catch up technologically, instead of having an informed data strategy and a well thought-out execution plan.

When that happens, data scientists face these choices: 1) become de facto analysts, taking on mostly ad-hoc tasks, 2) swim against the current and attempt to stitch together scraps of data to build one-off models, 3) become de facto engineers and attempt to build the missing platform, or 4) leave the company. Unfortunately none of these choices are desirable and lead to wasted time, resources, and potential.

(3) Instrumentation, shminstrumentation! Pitfall: deciding to push features to production and postponing instrumentation for a later date.

Lack of instrumentation is the #1 enemy of all-things-data. Oddly enough, data people are still fighting this battle, lobbying with product teams to take the time and instrument new features and functionality before they go to production. Most often, instrumentation doesn’t catch up with product functionality and if it does, it is superficial and not well designed. This leads to incomplete and low quality data, which drastically reduces the ability to do personalization.

The only way to get accurate, responsive, context-aware personalization models is to obsessively instrument your products — as they are being developed. That’s how you understand your users, their preferences, their hesitations, and their actions.

Before machine learning, before analytics, before hiring a brigade of data scientists for personalization: instrument your products!

(4) The hero data scientist. Pitfall: calling one-off, bailing-wire-and-duct-tape personalization a success.

Sometimes you need to go deep before you go broad. Build a few prototypes, surface problems, understand what’s needed and what works, and then … take a step back to solve the problem at scale. Quite often this last step is missing.

Sometimes by necessity, data scientists and engineers develop deep, one-off solutions for personalizing a very specific product feature or service. Once it is pushed to production, often there seems to be no time to generalize the work to scale from personalizing 10 product features to personalizing a thousand. This pattern occurs in all aspects of development: in content generation, presentation layer, app feature and structure, data and services, fraud detection, etc.

Successful personalization is not about building one-off solutions, creatively and against all odds. It is not about stitching together low quality data from five heterogeneous sources, writing tailored data processing code, spending countless nights building a hyper-optimized model, and lobbying with product managers so you can push your work to production. If you find yourself in that mode, you are likely missing the necessary support from the company leadership.

Instead, declare success when you have a platform in place, when relevant data is available quickly and consistently, when training, testing, and deploying models are repeatable and straight forward, when getting to production is a well traveled path.

(5) Personalization is a problem to be swarmed. Pitfall: having too many cooks in the kitchen, leading to organization paralysis.

In larger companies, there comes a point when personalization is declared a critical initiative. As soon as that happens, an uncoordinated myriad of people and teams start swarming the problem. Lack of clear responsibility and a weak mandate are par for the course. This situation often leads to product and architecture design by committee (a very large committee) and to HiPPO decision making. Useful work comes to a grinding halt and experts spend most of their time educating a large group of interested parties, which may or may not end up contributing.

The solution is as straightforward as stating the problem: clear focus, strong mandate, and executive support. Assembling a team that has previously built successful personalization platforms doesn’t hurt either.

(6) The Cloud provideth. Pitfall: declaring that Amazon or Google will provide a complete, end-to-end personalization solution that you will “just use”.

There is no reason to reinvent the wheel. It makes sense to use commoditized components and services as building blocks, while you focus most of your effort on the value that you and you alone can create.

Sometimes this principle gets hijacked and applied to more complex mechanisms that have not yet been standardized and commoditized, such as personalization, fraud detection, end-to-end security, auditing, etc. After a few failed attempts at building personalization platforms, it is not uncommon to see companies declare soft victory by strategic positioning: “actually, we shouldn’t build it anyway: Google, Amazon, etc. are working on a solution that we’ll just use”.

The Cloud does indeed make your life easier by providing data streaming infrastructure, parallel computation environments, machine learning building blocks. Still, for now it is up to you to build a coherent platform that works for your needs. There is a myriad of architectures for personalization and many of them are vertical-specific, product-optimized, and infused with domain knowledge. Today’s users expect apps and products to serve them, to adapt to them, and to delight them. You cannot afford to put off personalizing your products, keeping your fingers crossed, until the field matures and personalization becomes a stable, proven utility.

(7) It will be yuge, I promise. Pitfall: having a grand long-term vision with rigid execution and no plan for delivering incremental value.

It pays to have a clear and inspiring long-term personalization vision, with a solid design, and a well-thought out execution plan, but this is not enough. In my engagements I’ve seen personalization efforts fail because they do not take into account the fluid reality and ever-changing context. Without losing sight of the larger goal, it helps to be flexible, re-think milestones that generate useful deliverables, and solve smaller problems you encounter along the way.

Without significant value being delivered incrementally, it is quite reasonable for observers and sponsors to start asking tough questions: is the end in sight? will the outcome justify the effort and resources? is personalization just a distraction? is there a better way to get there?

On the human side, fatigue and uncertainty creep in when waiting too long for big, monolithic deliverables. Frequent small wins can help garner more support and enthusiasm. On the business side, such a large investment is more palatable when you see useful periodic deliverables and continuous strong evidence indicating that the personalization effort is on the right track.

Looking ahead

With such creative ways to fail, let’s remember that personalization is ultimately a large initiative that can also fail in all the traditional ways. Lack of vision, poor product definition, or navel-gazing can put the brakes on the project even before it gets started. Deficient design can be just as dangerous as poor execution and it can tank the personalization effort.

For users: interacting with personalized products and experiences is becoming the norm. For companies: investing in a personalization platform that can support every feature seamlessly is now a must. That is what’s needed to be able to compete and thrive.

You’ll have to navigate some treacherous waters to get there, but it’s well worth it. Once you have a well-oiled personalization machine in place***, you won’t be able to imagine evolving your product without one.

** don’t forget to also explore, not just exploit!

*** use personalization responsibly: privacy, ownership, and consent most definitely apply.

___

If you enjoyed this post, click the👏 below so other people can read it here on Medium.

Yoyo Labs is a premier Data & AI consulting firm. We specialize in building custom, state-of-the-art, high-impact data solutions. Our clients span verticals (Advertising, FinTech, Healthcare, Social, Manufacturing, Mental Health, etc.) and sizes (nonprofits and startups to Fortune 100). We care about product and we bring our deep data expertise and a relentless quality focus to bear. The more complex, large scale, intractable, gnarly data product problem you have, the better! It’s right up our alley. Talk to us! We can help.

Lucian Lita is founder of Yoyo Labs, previously founder of Level Up Analytics and data leader at BlueKai, Intuit, and Siemens Healthcare.