If you find yourself in this predicament, consider this: your hiring criteria may be off and your candidate pool may be too narrow.

When you focus on exact matches, the talent / candidate funnel is quite unforgiving: there are very few people out there with the perfect skill set overlap to begin with. Fewer of those also have the exact experience you (think you) require. Fewer still are good at what they do, smart, nice to work with, diligent, etc. And even fewer prefer your company over all of their other options.

If you don’t like these odds, perhaps it’s time to adjust your criteria and broaden the candidate pool. Find smart, dedicated, driven people with a strong foundation. Extend them the opportunity, teach them, let them teach you, and they’ll become the experts you need in no time.

So how do you broaden the candidate pool in practice? Glad you asked. Here are just three of the strategies I’ve successfully used in the past.

Skills: Tech Can Be Learned

Find people in your field with expertise in a slightly different area — they will initially have only a subset of the skills you require.

In the beginning of 2009, I was starting to build BlueKai’s engineering and analytics team, responsible for designing and developing the company’s big data platform. In just a few years we scaled to over 2 billion unique user profiles and to tens of trillions of events and transactions per month. It was the early days of commoditized, massive scale, distributed systems and at the time Hadoop was the way to go. Without thinking twice about it, I set out to hire experts in this ecosystem, namely “Hadoop engineers.” Silly me.

The problem was that except a for few folks at Google and Yahoo, there weren’t many “Hadoop engineers” out there. I searched everywhere, left no stone unturned, and in the process rejected many amazing back-end and full-stack engineers because of their lack of experience with Hadoop.

After nine months of failing miserably, I realized that my mistake was equating (a) success in the role of platform engineer with (b) knowledge of a specific technology. I had been looking for people with Hadoop experience, when I should have been looking for strong engineers with the passion to learn and the focus to deliver.

This experience radically changed the way I approached hiring: recognizing that skills can be acquired, considering a broader candidate pool, and identifying true predictors of success in a given role. That year I hired some of the best people I’ve ever worked with — none of them Hadoop engineers. They became experts in no time and together we built one of the best platforms around.

Foundation: Find Knowledgeable Neighbors

Broaden the pool even further to include experts in neighboring fields with a solid foundation, who have proven they can acquire skills such as the ones you need.

During my Ph.D. I went through a couple of transformative internships at IBM TJ Watson, in the applied machine learning (ML) and natural language processing (NLP) group. This is the group that earlier had laid the practical foundations for modern statistical machine translation in the late 1980s, revolutionizing the field. The same group produced the large scale NLP, information extraction, and overall machine learning behind the Jeopardy-winning Watson. As I started to interact with scientists and engineers, I noticed that many of the NLP, speech generation, and ML experts had formal backgrounds in electrical engineering and physics, rather than computer science. Not only were they great at their craft, but they also had perspective, coming from a different field.

Years later, I started my own company, Level Up Analytics, together with a couple of good friends, both of whom also have Ph.D.s in physics. We ended up hiring quite a few people with backgrounds in electrical engineering, physics, astronomy, space science, economics, etc. These were some of the best engineers, data scientists, and product managers I’ve seen, all successful in their roles. They built their computer science careers on a foundation of math, engineering, critical thinking, heavy-duty data processing, and coding. On top of that, they also had the passion and the drive to build and deliver good products. As for tools and technologies, they quickly and thoroughly learned what they needed along the way.

If I learned anything from these experiences is that it pays to broaden the pool of candidates by considering neighboring disciplines. For example for data science consider candidates who are hands-on Ph.D.s and postdocs in math, sciences, and engineering fields. They have the math and statistics background, the focus, and the depth. If you also test for practical problem solvers with good coding skills, you’re already ahead. For data engineering, consider looking at electrical engineers and embedded systems engineers — they already love optimizing, performance, and depth.

This really works. Insight Data Science, a wonderful company I advise, proves this every day. Its fellowship program identifies and brings powerful and creative minds from neighboring disciplines into data science, data engineering, health data, and AI. They go on to become key experts at startups, as well as established companies. In a sense, for data-related tracks, Insight helps you tap into a broader the pool to find great candidates, while reducing your effort and risk.

Geography: Zoom Out

Broaden the pool to include experts who live somewhere else— either bring them to you or take the job to them.

Before the ink was dry on my diploma, I was recruited by a quasi-startup, a visionary, move-mountains kind of group, within Siemens Healthcare. We were asked to bring intelligence — i.e. machine learning — to products and see them through to market. A couple of projects were were particularly ambitious: one on faceted search for medical records and one on active learning for information extraction from noisy documents. With limited resources and a strong need for data scientists and engineers, we iteratively broadened the candidate pool. Soon after, the team expanded to: front-end engineers in Princeton who learned new, advanced libraries on the job; app developers in Bangalore who had never before worked in search; medical domain experts in Philly who did not have prior experience in relevance ranking; back-end engineers in Mountain View, who relocated from Germany and took on the novel task of developing deployment services for machine learning. Fast-forward to today and many of us work together, now at Yoyo Labs.

Taught (not very gently) by scarcity and necessity, I ended up using geography to my advantage in every team, finding or growing experts across the US (Bay Area, Seattle, Philly, Boston, etc.) and internationally (India, Germany, Australia, Venezuela, UK, New Zealand, etc.).

And why not look across the world to broaden your candidate pool? Either create the opportunity for incredible people to relocate and join your merry band, or learn how to build, nurture and manage distributed teams. We’re currently doing both at Yoyo and looking at the people we work with, we couldn’t be happier with our approach.

When it’s all said and done, the opportunity cost is too great to be comfortably passive. Find smart, creative, and collaborative people with a solid foundation and then inspire them and watch them soar. Grow your own experts!

___

If you enjoyed this post, click the👏 below so other people can read it here on Medium.

Yoyo Labs is a premier Data & AI consulting firm. We specialize in building custom, state-of-the-art, high-impact data solutions. Our clients span verticals (Advertising, FinTech, Healthcare, Social, Manufacturing, Mental Health, etc.) and sizes (nonprofits and startups to Fortune 100). We care about product and we bring our deep data expertise and a relentless quality focus to bear. The more complex, large scale, intractable, gnarly data product problem you have, the better! It’s right up our alley. Talk to us! We can help.

Lucian Lita is founder of Yoyo Labs, previously founder of Level Up Analytics and data leader at BlueKai, Intuit, and Siemens Healthcare.

Grow Your Own Experts was originally published in Yoyo Labs Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Distributed Worker Queue

Distributed Worker Queues are a valuable decoupling mechanism for various use-cases in software & systems engineering:

• Decoupling of job production and job consumption, e.g. for tasks where human interaction is needed.
• Minimize latencies in user facing systems by moving resource intensive tasks to a designated set of machines, e.g. for video encoding.
• Purpose-built computational resources are required for a specific task, e.g. using GPUs to train a machine learning model.

Enter a database-backed distributed worker queue mechanism that is easy to test, deploy, and maintain, as well as reliable and cheap.

MySQL for a Distributed Worker Queue?

MySQL v8.0.1 introduced (finally 🥳) a SKIP LOCKED option to SELECT … FOR UPDATE and SELECT … FOR SHARE statements. Thus, a locking FOR UPDATE read that leverages the new SKIP LOCKED option will not wait to acquire a row lock, but instead, will simply ignore locked rows in the result.

Combined with transactions this becomes a handy feature to implement a simple, reliable and efficient distributed worker queue. Let’s dive into a possible solution.

Table Schema

CREATE TABLE IF NOT EXISTS worker_queue
(
    id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    command TEXT NOT NULL,
    is_done BOOLEAN NOT NULL DEFAULT FALSE,
    failure_count SMALLINT DEFAULT 0
) ENGINE=InnoDB DEFAULT CHARSET=UTF8MB4;

We are going to introduce an auto-increment id field. As follows, sorting in ascending or descending order on the id field will enable either FIFO or LIFO queue characteristics. In the given example the command is a simple text field, but it could be anything fitting your needs. In order to be able to indicate that a specific message in the queue has been processed the is_done field is introduced. Even retry and maximum retry capabilities can be realized through a simple failure_count.

Adding a Message

Adding a new message is literally as simple as an INSERT into a table.

INSERT INTO worker_queue (command) VALUES (‘do this’);

Retrieve & Lock a Message

START TRANSACTION;

SELECT id, command FROM worker_queue
WHERE failure_count < 3
AND is_done = FALSE
ORDER BY id
LIMIT 1
FOR UPDATE SKIP LOCKED;

A worker process will start a transaction which will stay open until the corresponding work is done. In our case we sort by id in ascending order to implement a FIFO. The given query will read and obtain a read-lock for one row, this keeps other worker instances from processing the message simultaneously. The query also filters rows which are currently locked using the newly introduced SKIP LOCKED option. Additionally failure_count < 3 will filter for rows which have not been consumed and failed for more than three times. We will cover this in more detail shortly. Once a lock is obtained, we can start processing the respective message. In case a worker process dies, the lock for a given message will be released automatically since the lock was obtained in a not yet committed transaction. This way the message will be available for consumption again.

Acknowledge a Message

To acknowledge that a message has been processed, is_done has to be set to true as final statement of the current transaction.

UPDATE worker_queue 
SET is_done = TRUE 
WHERE id = <id>;

COMMIT;

Graceful Failure & Maximum Retries

Dead Letter Queue behavior can be achieved through the previously introduced failure_count. When a failure count for a message exceeds a certain threshold it will no longer be returned from the earlier query. A separate worker process which obtains such elements will help analyze and understand why messages are failing.

UPDATE worker_queue
SET is_done = FALSE, failure_count = failure_count + 1
WHERE id = <id>;
COMMIT;

Cleaning up

Since we are filling up the worker queue table with messages it is desirable to periodically clean up completed or faulty tasks. This could be accomplished through a simple delete:

DELETE FROM worker_queue
WHERE is_done = TRUE
OR failure_count >= 3;

If the history or an archive is needed, messages could be moved to different tables instead. MySQL’s Event Scheduler could be used to implement periodic query execution.

Conclusions

Dozens of strategies and technologies exist for implementing a distributed worker queue. If you favor PostgreSQL for example, you could use the same strategy as of version 9.5.

Using the database’s internal lock mechanism has some advantages:

• MySQL is extremely common. Almost everyone has a MySQL or PostgreSQL installation up and running somewhere. If not, it is straightforward to set up. If you don’t want to deal with the setup yourself you could use managed services e.g. from any of the top cloud providers.
• Since we are using MySQL internal locking, producer and consumer are language agnostic. All you need is a database driver in the programming language of your choice. Furthermore, in a heterogeneous environment where consumer processes are implemented in different languages, using a standard database for coordination can become the lingua franca of your distributed workers.

Without a doubt, this approach is not made for “web scale” type of workloads. Nonetheless, since we are using database internal locking mechanisms, the throughput is high enough for a large class of workloads with hundreds or thousands of work items per second.

Distributed Worker Queues — A piece of Cake🍰 was originally published in Yoyo Labs Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Origin: The Three Laws of Robotics

In 1942, Isaac Asimov introduced the “Three Laws of Robotics”, a set of absolute rules that every¹ Artificial General Intelligence (AGI)² robot-servant in his science fiction stories must obey. In short, the rules are (1) Don’t let humans be harmed, (2) Obey humans, and (3) Protect yourself.

The three laws are hierarchically ordered, so lower-numbered laws override higher-numbered laws whenever the laws are in conflict: for example, any action (or inaction) taken to obey the Second Law cannot interfere with obeying the First Law (but it can interfere with obeying the Third Law).

After exploring scenarios where AGI robot-servants interact with each other and society at large, Asimov added a fourth, or “Zeroth”³ Law, which can be summarized as: (0) Don’t let humanity be harmed.

Think what you will about how applicable these laws are to AGIs⁴; the Three Laws of Robotics do offer sound fundamental requirements for safely designing systems that serve our needs, while also encouraging the vital discussions and definitions that are needed to adequately implement those requirements.

Adapting the Three Laws to Personalization

The Three Laws of Robotics were formulated as a set of instructions that were ‘hard-coded’ into every AGI robot-servant as rules for governing its behavior. Like a human, an AGI robot-servant could absorb the context of the situation it found itself in and then apply the laws to decide what to do next.

To date, the best personalized services are only Artificial Narrow Intelligence (ANI)² systems, nowhere near AGI-level sophistication. Despite how advanced they may seem, ANI systems are not capable of understanding the Three Laws of Robotics, because they inherently lack the ability to understand the context of any circumstance that is outside of what they have been explicitly programmed to do. In other words: ANI systems are roughly as self-aware as a microwave oven.

It is increasingly important for us to understand ANI system limitations and issues, such as unintended bias, because we are trusting ANIs to make increasingly important decisions, including who gets a job interview, who gets approved for a loan, who is a suspect for a crime, how long a convicted criminal is sentenced, and who is granted parole.

Because personalized services and ANI systems cannot think for themselves, the humans who build those systems must think for them.

Thus, the Laws of Mindful Personalization are intended for the humans who create personalized services, as guiding principles for how to create and maintain ethical and positive products.

With that in mind, let’s specialize the Laws of Robots to personalization.

Law #1: Don’t let humans be harmed → Don’t harm the user.

From an ethical perspective, it’s simple and obvious to say, “Don’t harm the user”. From an implementation perspective, it is very difficult to define what “harm” is for every user, because every user has unique circumstances and needs.

Let’s start with an example of how can a personalized service harm a user: Take Apple iPhones, which include Siri, an ANI personal assistant, as well as a plethora of apps and media that Apple personalizes for its users. Through personalization and design, Apple tempts its customers to use their iPhones frequently. However, Apple found that it has been overly successful because users, especially children, were getting addicted. While we can all agree that “addiction” is bad, the point at which “normal use” ends and “addiction” begins is difficult to define, and highly subjective.

It is important to note that well-implemented personalized services will tend to be addictive, by design. While a person’s well-being is generally his own responsibility, an addict cannot be trusted to make the best decisions with regards to his addiction; therefore, it is the responsibility of the business that offers the personalized services to do what it can to make sure its users are not being harmed.

How? Businesses can protect their users by actively responding to user concerns as they arise, actively ensuring that the personalized services are supporting people’s well-being, and actively checking for bias in their ANIs’ models and data. Businesses can also prevent harm from third-parties by guarding their users’ privacy. It isn’t possible to foresee all the ways in which a personalized service may harm a user, but it certainly is possible to minimize harm by acting as soon as any issue arises. Businesses who succeed at fulfilling Law #1 can look forward to having long-lived, happy, and functional customers who can continue to buy their products long into the future.

Law #2: Obey humans → Do what the user wants.

Personalized services are intended to delight you by figuring out what you want and offering it to sell it to you. The key here is that the personalized services need to figure out what you want; in other words, personalization must not be used to manipulate you into buying something you don’t actually want.

For example, a user may decide to entrust a personalized service to purchase recurring annual gifts for her loved ones. Applying Law #2 means that the personalized service must carry out the task given to it by the user (e.g., pick out the most awesome gifts), and not a different task set by the business offering that service (e.g., spend as much of the user’s money as possible).

Similarly, personalization must not be used as a vehicle for discrimination: no one wants to be treated poorly.

How? Businesses can be responsive to their users by developing personalized services that strive to provide value to all of their customers. The upside of this strategy is that products developed this way are more likely to become popular, to receive positive press, and to boost your business’s reputation.

Law #3: Protect yourself → Monetize effectively. This one is easy: companies won’t offer personalization unless it positively impacts their bottom line. Nevertheless, it’s worthwhile to explicitly reserve that right, because history has shown that companies can be forced to do things that aren’t good for anyone⁵.

How? Businesses can monetize effectively by investing in a sound data strategy with good metrics, efficient and scalable data infrastructure, advanced machine learning and ANI models, and agile deployment practices. This upfront investment can pay for itself many times over, as it attracts new customers, strengthens existing relationships, and keeps your business relevant into the future.

Like the Laws of Robotics, the Three Laws of Mindful Personalization also require a fundamental, zeroth law to prevent societal disaster:

Law #0: Don’t let humanity be harmed → Don’t let society be harmed.

In other words: don’t let the collective set of individuals using personalization result in an avoidable, bad situation. Recent revelations that YouTube is unintentionally enforcing radicalism, Facebook is creating ideological echo chambers to maximize ad revenue, and Microsoft’s facial recognition software was inadvertently racist underscore the absolute importance of this theme, as well as the moral, business, and technical challenges associated with fixing these problems as they arise.

How? Businesses can prevent harm to society by actively monitoring for potential issues, responding to issues as they arise, leveraging scientific findings and experiments to develop effective solutions, protecting user data, and having the courage to do what’s right, even if it costs a little bit more to do it that way. The upside of supporting the zeroth law is having a positive reputation in a functional and stable society. The risk of neglecting the zeroth law is arousing the public’s wrath.

The Three Laws of Mindful Personalization

In summary, the Three Laws of Mindful Personalization are:

1. Don’t harm the user.

2. Do what the user wants, unless this conflicts with the First Law.

3. Monetize effectively, unless the way you are monetizing conflicts with the First or Second Laws.

And finally, the one law to rule them all:

0. Don’t let society be harmed.

I’d love to hear what you think of this fresh twist on a familiar topic — it’s as important to have meaningful discussions on what the laws mean and how we’d implement them, as it is to have discussions on what the laws ought to be.

Please leave a comment below to continue the discussion. And, if you enjoyed this thought-piece, please share it with a friend, and applaud as you see fit.

— — —

Footnotes:

1. This is a generalization: die-hard Asimov fans may point out that there technically are three exceptions; however, none of those three are core to Asimov’s universe(s), so suffice to say: the vast majority of Asimov’s relevant works assume that the Three Laws are fundamentally required.

2. Some useful definitions + clarification:

• AGI = “Artificial General Intelligence”, which is defined as a single computing system that can do any intellectual task that a human can do. These mythical, powerful systems are frequently viewed as both the means to a futuristic utopia and the harbinger of inevitable doom for humanity.
• ANI = “Artificial Narrow Intelligence”, which is defined as a single computing system that can accomplish one narrow task or narrow set of tasks that we previously thought only humans could do. This includes machine learning- and deep learning-based algorithms, which are specialized to a certain task, like AlphaGo, or Google Search.

Note that both of these definitions are strongly time-dependent: technologies that would be classified as an “ANI” 15 years ago are considered “too basic” to be called an ANI today.

To illustrate the huge difference in capability of an AGI vs an ANI, here are two examples using state-of-the-art (2018) ANIs:

(1) Skynet from the Terminator movies (AGI) vs your personalized Netflix feed (ANI). Note that I’m not making fun of Netflix here — they’re doing really cool stuff in personalization.

(2) Data from Star Trek: The Next Generation (AGI) vs {Duplex, Siri, Alexa} (ANI).

• Data physically looks just like a human, complete with gestures and facial expressions. “Data’s physical and mental capabilities were far superior to that of virtually any organic or cybernetic humanoid” in the Star Trek universe.
• Google’s Assistant AI Duplex can sound like a real human and make a phone call to set a hair appointment … but probably isn’t successful every time, which is why it’s not yet publicly available.
• Apple’s Siri can take dictation. After six years of using Apple’s Siri to dictate nearly all of his emails and text messages, my friend still can’t get Siri to spell his name correctly.
• Amazon’s Alexa can take and place orders on Amazon.com, and within the last year, it has learned to recognize voices, so [if your host set that up] you can no longer walk into a friend’s house and prank-order two tons of creamed corn.

3. While this is technically fudging things so Asimov wouldn’t have to re-number his previous laws, I’m sure that computer scientists in the audience will appreciate Asimov’s decision to implement proper zero-based numbering. Some refer to the amended laws as Asimov’s “3+1 Laws of Robotics”, or the “Four Laws of Robotics”; but let’s be honest: the “Three Laws of Robotics” sounds best.

4. It’s been posited that the Three Laws of Robotics are not ethical to apply to a fully-sentient being because they fundamentally assume that the beings they are applied to should be treated as less than human. While that is a reasonable argument to bring up with regards to beings like Data from Star Trek, or Andrew in The Bicentennial Man, that doesn’t apply to today’s machine learning and deep learning algorithms, which are still approximately as “sentient” as a desktop calculator.

5. Example: General Motors was forced to pay employees to not work, which was expensive for GM, and depressing for their employees, who wanted to work but couldn’t (in order to be paid).

The Three Laws of Mindful Personalization was originally published in Yoyo Labs Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

You may have heard of the condition called mirror muscle syndrome: it occurs when people focus on training the muscles that they can see in the mirror, like their abs, chest, and biceps, while neglecting the opposing back and leg muscles. This lopsided focus leads to muscle imbalances that result in terrible posture, pain, and performance plateau.

We’ve all seen the front-end/back-end imbalance in tech: the beautiful website that takes a full minute to load, the latest smart watch that can’t figure out that you got off the elliptical machine an hour ago, and the AI virtual assistant that can’t spell your name. These are all cases where companies have poured their efforts into what’s visible and neglected the other side of their products, leading to disappointed customers and missed sales.

With the rise of personalization, we now find ourselves with a new mirror muscle syndrome imbalance: between personalization and privacy. Personalization is what customers immediately see when they use your app or visit your website. It’s in the recommendations you offer, the ads you show, and the content you display. Privacy is the supporting back muscles to personalization — it’s what gives customers the sense of trust needed for them to share the data required for personalization to work. It’s what makes customers feel that your personalized services are delightful, and not creepy.

The stakes in this case are even higher: customers want both personalization and privacy, and they’re willing to pay for it. On the flip side, not complying with regulations that protect privacy, such as GDPR, can lead to massive fines, and customers are ditching companies they don’t trust.

Traditionally, investment in data privacy has lagged behind investment in personalization, because companies were not incentivized to care about data privacy. This is no longer the case. In the US, GDPR-like legislation is being proposed by multiple members of Congress and industry, and is also considered by the White House. In India, a new data privacy law is being drafted as well. It’s time to get serious about privacy.

The first step in every improvement program is to “recognize that there is a problem.” To help you figure out if you have a gap in privacy, personalization, or both, we’ve devised a simple test that is independent of company maturity, size, and vertical: the five-point muscle test.

The Five-Point Muscle Test

We’ve defined five “muscles” (qualities) to focus on, where you need to be “strong” in both data privacy and personalization. The examples below will help you assess your current strength. As you follow along, circle the box in each column that best describes where your business is now.

#1 Automation: Technology is supposed to automate repetitive tasks
… so let it.

#2 Robustness: Technology can be overwhelmed, fail, or be attacked.
Are you prepared?

#3 Timeliness: There’s a minimum speed to play.

#4 Transparency: Everyone knows what’s going on — no surprises.

#5 Security: Claiming privacy without security is like using a screen door as an airlock on a spaceship.

So, how’d you do?

Is your Privacy as strong as your Personalization? Are you balanced across your muscles, or do you have mirror muscle syndrome?

If you’re strong in both privacy and personalization across all five muscles — Bravo! You are one of a small number of companies who really know what they’re doing. Amazon is less likely to be eating your lunch anytime soon and your users will thank you for protecting their data.

If you find that you’re not as strong as you’d like across any of these muscles, in either privacy or personalization, invest now to bridge the gap and find your balance. Consider getting experienced outside help. In weightlifting, progress comes much faster and more easily with a knowledgeable personal trainer at your side. The same goes for tech.

Final Thoughts

Privacy and personalization are fundamentally related because both require a smart data platform that is automated, robust, timely, transparent, and secure.

Businesses have typically invested more in personalization because of the obvious incentives they see in the mirror, like higher engagement and better conversion rates. With increased scrutiny and changes in the privacy landscape, businesses with mirror muscle syndrome are hitting a performance plateau, as customers leave or withhold their data, due to lack of trust. In order to push past this plateau, businesses need to strengthen their privacy “muscles”, by investing in equivalent capabilities to personalization.

When both privacy and personalization are strong, customers are happy and willing to share their data, which improves your product, and sets up a virtuous cycle that generates increased revenue, customer loyalty, and brand recognition.

____

Yoyo Labs is a premier Data & AI consulting firm. We specialize in building custom, state-of-the-art, high-impact data solutions. Our clients span verticals (Advertising, FinTech, Healthcare, Social, Manufacturing, Mental Health, etc.) and sizes (nonprofits and startups to Fortune 100). We care about product and we bring our deep data expertise and a relentless quality focus to bear. The more complex, large scale, intractable, gnarly data product problem you have, the better! It’s right up our alley. Talk to us! We can help.

About the Authors

Lucian Lita is founder of Yoyo Labs, previously founder of Level Up Analytics and data leader at BlueKai, Intuit, and Siemens Healthcare.

Margot Kimura is Lead Data Scientist at Yoyo Labs, previously a principal investigator of cybersecurity and decision-making products for the federal government at Sandia National Laboratories.

Five Strengths to Build Across Privacy & Personalization was originally published in Yoyo Labs Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Umm … not yet, but they’re working on it!

Aah, that feeling when personalization gets it right: a news feed that provides relevant updates, a virtual assistant that learns your interests, an app that changes its workflow based on your repeated usage. It’s a magical feeling. Unfortunately, it is still a rare feeling. The reality is that most products are thinly personalized, poorly personalized, or not personalized at all.

If personalization is the best thing since sliced bread**, why exactly is it notoriously missing from many of the products and services we interact with on a day to day basis? Why is it not imbued into everything from user experience to content generation, making our personal and professional life better? It turns out that getting to personalization at scale is not that easy.

Over the past decade I’ve been working with companies in several verticals — fintech, healthcare, social, advertising, security, communication, infrastructure, etc. — helping clients build products and platforms spanning the personalization spectrum. In my engagements I noticed a handful of misconceptions that are responsible again and again for stalling, obstructing, or derailing sometimes massive efforts to get personalization off the ground. I’m sharing some of these misconceptions here.

(1) Machine learning equals Personalization! Pitfall: believing that machine learning (or more amorphously “AI”) alone is all it takes to do personalization.

Yes, machine learning is the engine that makes personalized experiences possible. Yet it takes a lot more to achieve personalization at scale: product instrumentation, timely data collection, fast data processing and feature computation, secure and flexible infrastructure to scale to hundreds or thousands of models in development and production, automation, security, etc.

To reach personalization at scale, it is quite common to spend your early time and energy on platform, data, and user feedback integration, rather than on machine learning algorithms. However, this reality is rarely reflected in most plans for personalization. Instead, the discussion is centered around ML capabilities: e.g. “so … will we be able to do deep learning?”

One thing is clear: AI will not subjugate humans without a good platform, solid integration, and thorough automation.

(2) Hire 99 data scientists! Pitfall: building a large data science team for personalization too early, well before you can leverage its talent and skill set.

In their desire to personalize products, companies sometimes rush to hire disproportionately large numbers of data scientists early on. This can be quite disastrous if they do not have a solid and scalable data platform — from consistent instrumentation all the way to flexible access to data and computation. In most cases it is a knee jerk reaction in an effort to catch up technologically, instead of having an informed data strategy and a well thought-out execution plan.

When that happens, data scientists face these choices: 1) become de facto analysts, taking on mostly ad-hoc tasks, 2) swim against the current and attempt to stitch together scraps of data to build one-off models, 3) become de facto engineers and attempt to build the missing platform, or 4) leave the company. Unfortunately none of these choices are desirable and lead to wasted time, resources, and potential.

(3) Instrumentation, shminstrumentation! Pitfall: deciding to push features to production and postponing instrumentation for a later date.

Lack of instrumentation is the #1 enemy of all-things-data. Oddly enough, data people are still fighting this battle, lobbying with product teams to take the time and instrument new features and functionality before they go to production. Most often, instrumentation doesn’t catch up with product functionality and if it does, it is superficial and not well designed. This leads to incomplete and low quality data, which drastically reduces the ability to do personalization.

The only way to get accurate, responsive, context-aware personalization models is to obsessively instrument your products — as they are being developed. That’s how you understand your users, their preferences, their hesitations, and their actions.

Before machine learning, before analytics, before hiring a brigade of data scientists for personalization: instrument your products!

(4) The hero data scientist. Pitfall: calling one-off, bailing-wire-and-duct-tape personalization a success.

Sometimes you need to go deep before you go broad. Build a few prototypes, surface problems, understand what’s needed and what works, and then … take a step back to solve the problem at scale. Quite often this last step is missing.

Sometimes by necessity, data scientists and engineers develop deep, one-off solutions for personalizing a very specific product feature or service. Once it is pushed to production, often there seems to be no time to generalize the work to scale from personalizing 10 product features to personalizing a thousand. This pattern occurs in all aspects of development: in content generation, presentation layer, app feature and structure, data and services, fraud detection, etc.

Successful personalization is not about building one-off solutions, creatively and against all odds. It is not about stitching together low quality data from five heterogeneous sources, writing tailored data processing code, spending countless nights building a hyper-optimized model, and lobbying with product managers so you can push your work to production. If you find yourself in that mode, you are likely missing the necessary support from the company leadership.

Instead, declare success when you have a platform in place, when relevant data is available quickly and consistently, when training, testing, and deploying models are repeatable and straight forward, when getting to production is a well traveled path.

(5) Personalization is a problem to be swarmed. Pitfall: having too many cooks in the kitchen, leading to organization paralysis.

In larger companies, there comes a point when personalization is declared a critical initiative. As soon as that happens, an uncoordinated myriad of people and teams start swarming the problem. Lack of clear responsibility and a weak mandate are par for the course. This situation often leads to product and architecture design by committee (a very large committee) and to HiPPO decision making. Useful work comes to a grinding halt and experts spend most of their time educating a large group of interested parties, which may or may not end up contributing.

The solution is as straightforward as stating the problem: clear focus, strong mandate, and executive support. Assembling a team that has previously built successful personalization platforms doesn’t hurt either.

(6) The Cloud provideth. Pitfall: declaring that Amazon or Google will provide a complete, end-to-end personalization solution that you will “just use”.

There is no reason to reinvent the wheel. It makes sense to use commoditized components and services as building blocks, while you focus most of your effort on the value that you and you alone can create.

Sometimes this principle gets hijacked and applied to more complex mechanisms that have not yet been standardized and commoditized, such as personalization, fraud detection, end-to-end security, auditing, etc. After a few failed attempts at building personalization platforms, it is not uncommon to see companies declare soft victory by strategic positioning: “actually, we shouldn’t build it anyway: Google, Amazon, etc. are working on a solution that we’ll just use”.

The Cloud does indeed make your life easier by providing data streaming infrastructure, parallel computation environments, machine learning building blocks. Still, for now it is up to you to build a coherent platform that works for your needs. There is a myriad of architectures for personalization and many of them are vertical-specific, product-optimized, and infused with domain knowledge. Today’s users expect apps and products to serve them, to adapt to them, and to delight them. You cannot afford to put off personalizing your products, keeping your fingers crossed, until the field matures and personalization becomes a stable, proven utility.

(7) It will be yuge, I promise. Pitfall: having a grand long-term vision with rigid execution and no plan for delivering incremental value.

It pays to have a clear and inspiring long-term personalization vision, with a solid design, and a well-thought out execution plan, but this is not enough. In my engagements I’ve seen personalization efforts fail because they do not take into account the fluid reality and ever-changing context. Without losing sight of the larger goal, it helps to be flexible, re-think milestones that generate useful deliverables, and solve smaller problems you encounter along the way.

Without significant value being delivered incrementally, it is quite reasonable for observers and sponsors to start asking tough questions: is the end in sight? will the outcome justify the effort and resources? is personalization just a distraction? is there a better way to get there?

On the human side, fatigue and uncertainty creep in when waiting too long for big, monolithic deliverables. Frequent small wins can help garner more support and enthusiasm. On the business side, such a large investment is more palatable when you see useful periodic deliverables and continuous strong evidence indicating that the personalization effort is on the right track.

Looking ahead

With such creative ways to fail, let’s remember that personalization is ultimately a large initiative that can also fail in all the traditional ways. Lack of vision, poor product definition, or navel-gazing can put the brakes on the project even before it gets started. Deficient design can be just as dangerous as poor execution and it can tank the personalization effort.

For users: interacting with personalized products and experiences is becoming the norm. For companies: investing in a personalization platform that can support every feature seamlessly is now a must. That is what’s needed to be able to compete and thrive.

You’ll have to navigate some treacherous waters to get there, but it’s well worth it. Once you have a well-oiled personalization machine in place***, you won’t be able to imagine evolving your product without one.

** don’t forget to also explore, not just exploit!

*** use personalization responsibly: privacy, ownership, and consent most definitely apply.

___

If you enjoyed this post, click the👏 below so other people can read it here on Medium.

Yoyo Labs is a premier Data & AI consulting firm. We specialize in building custom, state-of-the-art, high-impact data solutions. Our clients span verticals (Advertising, FinTech, Healthcare, Social, Manufacturing, Mental Health, etc.) and sizes (nonprofits and startups to Fortune 100). We care about product and we bring our deep data expertise and a relentless quality focus to bear. The more complex, large scale, intractable, gnarly data product problem you have, the better! It’s right up our alley. Talk to us! We can help.

Lucian Lita is founder of Yoyo Labs, previously founder of Level Up Analytics and data leader at BlueKai, Intuit, and Siemens Healthcare.

7 Sure Ways to Fail at Personalization was originally published in Yoyo Labs Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

“If you can’t explain it to a five year old, you don’t understand it well enough” said 50% Einstein, 40% Feynman, and 10% my need to make it #ELI5 compliant.

When helping companies with their personalization strategy, I often reach for simple and intuitive ways to describe the personalization spectrum. It sets up a common language and lays the foundation for a shared vision across business and technology. It also helps build a strong intuition around personalization for execs and sponsors, which reduces ambiguity and builds confidence.

Not surprisingly, I found that “in the field,” explaining personalization using food-based examples has good traction. As a silly challenge, I thought I’d take my food metaphor one step further and … explain it like I’m five.

Here is my #ELI5, ice cream-centric doodle of the personalization spectrum.

Gut Feel: I wonder what ice cream flavor people like. Give me your best guess. Strawberry you say? Ok, from now on people can only have strawberry ice cream!

— Many people are happy, many are not happy. It is easy to pick a flavor.

A/B Testing: I wonder what ice cream flavor people like. Give me your three best guesses. Chocolate, strawberry, or durian you say? Ok, let’s try them out on a few people; have them grab one of the three without looking. The ones who got chocolate seem a bit happier. Ok, from now on people can only have chocolate ice cream!

— Many people are happy, some are not happy. You can keep experimenting, make more people happy, but it’s still only one flavor.

Coarse Personalization: I wonder what ice cream flavor people like. Give me your three best guesses! Chocolate, strawberry, or durian you say? Our robot, Bender, watches people try out these flavors. After a while, he starts to guess which of the three flavors each person likes. He becomes a good guesser. From now on, if anybody wants ice cream, Bender will decide what flavor they get: chocolate, strawberry, or durian.

— Most people are happy, a few are not happy. It takes a while for Bender to become a good guesser.

Deep Personalization: People like ice cream in different flavors, in different sizes, and with different toppings. In a cup, in a cone, or on a stick. Our robot, Bender, watches many people try out many different kinds of ice cream. After a while, he starts to guess what kind of ice cream is each person’s favorite. From now on, if anybody wants ice cream, Bender will try to guess the best ice cream, just for them. And he’ll keep experimenting to refine his guessing skills.

— Many people are very happy, a few are not happy. It takes a long time and it’s hard for Bender to become a good guesser, but it’s worth it.

Author’s note: without data, your product may end up durian-flavored.

____

If you liked this doodle, click the👏 below so other people can read and enjoy it here on Medium.

Yoyo Labs is a premier Data & AI consulting firm. We specialize in building custom, state-of-the-art, high-impact data solutions. Our clients span verticals (Advertising, FinTech, Healthcare, Social, Manufacturing, Mental Health, etc.) and sizes (nonprofits and startups to Fortune 100). We care about product and we bring our deep data expertise and a relentless quality focus to bear. The more complex, large scale, intractable, gnarly data product problem you have, the better! It’s right up our alley. Talk to us! We can help.

Lucian Lita is founder of Yoyo Labs, previously founder of Level Up Analytics and data leader at BlueKai, Intuit, and Siemens Healthcare.

Personalization — Explain it Like I’m Five was originally published in Yoyo Labs Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.