If it Doesn’t Taste Good Alone, it Won’t Taste Better with A.I.
What are algorithms? How are they different? And how to find out which ones are truly valuable?
by Prof. Thales S. Teixeira
Algorithmic technologies — Machine Learning (ML), Artificial Intelligence (AI) and Big Data (BD) — have become all the rage as tools for business innovation. Startup entrepreneurs have come to find that simply adding ML or AI to their business descriptions, without having to explain the nuts and bolts of how it works, results in an immediate spark of interest from potential investors.
For the past three years, I have been a judge in CNBC’s annual Disruptor 50 competition, which aims to identify the most disruptive startups out there. Looking at this past year’s applications of startups, I have seen a sharp increase in descriptions based on these algorithmic technologies. Vague phrases such as “intelligence platform,” “sophisticated computer vision,” “deep learning algorithms,” and “highly innovative BD and AI” have become common parlance in the startup world. Add to this the fact that most people have no clue what these terms actually mean, what you get is a wave that everyone is trying to catch. It is not unlike the pizza market. As anyone who has visited New York City can attest, all pizza joints advertise that they have “the best pizza in town.”
So, what is an algorithm anyway?
Algorithms are Rules
The vast majority of ML, AI and BD algorithms used by digital startups and consumer-facing tech companies today actually have many elements in common. These can be divided into one of three classes based on their purpose: classification, prediction and automation algorithms.
Classification is intended to use data on various entities such as objects, people or businesses to classify each entity into a type. For example, Affectiva uses webcam recordings of a person’s facial expressions to classify their underlying emotional state into one of seven so-called universal emotions (happy, sad, angry, surprised, disgusted, fearful or neutral).
My first attempt to design a machine learning tool was in the early 2000s after graduating from college. I was hired by an executive recruitment firm to look at more than 100,000 surveys of executives and find correlations between personality tests and market value as measured by annual salaries (Finding: more ambitions, competitive and dominating execs earned more money). At the time, the technical terms for these tools were different. Yet the purpose and the underlying techniques were very similar. In fact, a recent survey among data scientists revealed that the top three methods used for machine learning today have been around for many decades. Multivariate regression analysis has been around for at least 200 years. Clustering analysis is 80 years old. Neural networks are about 70 years old.
Prediction algorithms are those that take data on entities and contextual factors in order to predict a future behavior and then influence it. StitchFix, a subscription-based clothing startup, uses data on people’s past purchases, likes, dislikes, sizes and surveys to determine what items of clothing each consumer is more likely to purchase.
During my doctoral studies at the University of Michigan, I created a novel prediction algorithm. I used eye-tracking technology to measure people’s eye movements as they watched ads and filmed their faces using so-called computer vision algorithms to gauge emotional responses from facial expressions. I built complex algorithms that would detect when people lost interest in an ad and were likely to skip or click away. Processing real-time video is much more data-intensive than personality surveys, but it uses the same basic principle: identify characteristics that correlate with a desired outcome. This originated my work in the field of Economics of Attention. Since then, a flood of new startups claim to uniquely use ML or AI to detect emotions from people’s faces. In reality, the algorithms are all very similar in nature.
Automation algorithms use contextual data to, not only predict, but independently act on it. Self-driving cars, for example, use contextual data gauged from radars and lidars (light-based radars) to determine where the car is, where it is heading and obstacles, using all this data to calibrate and correct its position, trajectory and speed. Goggle’s Waymo, Uber, and Tesla are currently experimenting with these algorithms. Initially, these algorithms start off as supporting human decisions, e.g., are semi-autonomous. Eventually, the very best become so robust that they reach autonomous status and replace humans altogether. But self-driving cars and robots are not the only areas that use automation.
In one recent application of AI algorithms, my colleagues and I showed a series of short movie clips to measure participants’ emotional reactions using their webcams. Learning what scenes in movie trailers “spark joy,” to borrow from Marie Kondo’s vernacular, helped us select the few scenes to put into a five or 10 second clip to show on Netflix’s website as a recommendation tool. In short, if viewers smiled when watching a scene, we show it to others in short auto-playing clips to increase the chances that a new person watches the movie. Nothing mind-blowing if you think about it. But because it is fast and automatic, engagement with new content has improved significantly for Netflix users.
Understanding the Value of an Algorithm
Since algorithms carry an inherent error rate — a classification algorithm can misclassify, a prediction can mispredict, and an automation one can make a wrong decision — the primary way to rigorously evaluate them is by measuring their error rates in the field, the costs associated with errors, and the reduction in the error as data grows. This is rather complex and should be done by experts.
But what if you are not a technical person? One trick is to have the people who designed the algorithm translate it into simple “IF-THEN” statements. No matter how complex an algorithm is, it can always be broken down into multiple (potentially infinite) input and output functions as such:
If {this is observed} THEN {classify/predict/do} that.
A classification algorithm, for instance, can say IF person’s lip corners increase in distance, THEN she is classified as feeling joy (because she might be smiling). A prediction algorithm could say that IF a person bought a red cardigan, THEN she is very likely to purchase a white top to go underneath. An automation algorithm for a self-driving car might determine that IF the lidar identifies a red hexagonal sign, THEN it should decelerate and apply the brakes to stop.
Any algorithm can be translated into anywhere from hundreds to billions of such IF-THEN statements. Therefore, going through all of them would be an impossible task. Rather than evaluating all possible scenarios, the idea is to judge the reasonableness of the outputs — the THEN clauses — by looking at the likely situations. Here, it is important to both evaluate most probable scenarios (the norm) as well as extreme, but still possible, scenarios.
But be careful. Do not let the builders of these algorithms solely control all the scenarios that you evaluate. When GM executives went out to Silicon Valley to see and assess the self-driving car programmed by Cruise Automation, a 3-year-old, 40-person startup, they all were shown the autonomous vehicles driving in the same stretch of Highway 101 again and again and again. In 2016, GM paid $1 billion for a company that has yet to launch a functioning commercial product.
Separating the Good, the Bad and the Ugly
Valuable algorithms are “smart” in that they have two defining characteristics: (1) They collect high quality input data for the IF clauses, and (2) they propose “reasonable” THEN clauses. As such, a quick reality check is to look at the type, amount and quality of the data as well as a sample of IF-THEN statements that the algorithm is based upon in order to judge its reasonableness. Take a look under the hood. Does the algorithm do things (classify, predict or automate) that you find reasonable or unreasonable? If confronted with the same data, would a reasonable person have done the same?
Bad algorithms make too many mistakes and are useless. One reason is these algorithms operate with very limited or unreliable data in the IF clause. For instance, using Google search in cognito often provides much worse results for finding a restaurant then if you are signed in and your location can be determined. The other way in which algorithms fail is when they propose very unreasonable THEN clauses, such as suggesting an Iranian movie to someone who has never watched a foreign movie or turning an autonomous car into the opposite direction of a one-way street.
Somewhat useful algorithms have the right data and would propose to do exactly what someone with that same data would do. In a sense, they are not smarter than humans — just faster, cheaper or effortless. While the press talks a lot about Amazon and Google, most of the innovative startups I see today make simple decisions quicker and more cost-effective thousands of times per day. Examples include, Blue River which assesses the health of a plant crop and decides whether or not to spray it with herbicide and Weathfront which invests its client’s money so as to minimize her tax burden. Nothing mind-blowing, but valuable nonetheless.
The most valuable algorithms, on the other hand, go beyond and do things that someone would not have thought of doing, but, upon further look at the data and the patterns, they would understand why the algorithm proposes those actions. An example of this is Google’s ML algorithm that manages energy used for cooling servers in its data centers. By letting it freely explore combinations of temperature, power and water pump usage, the ML algorithm was able to reduce relative electricity usage by 40 percent compared to human-made decisions. Farmers Business Network, Rent the Runway, The Good Face Project, and TVision Insights are some of my favorite startup examples of algorithms going above and beyond.
These ‘intelligent’ algorithms learn by extrapolating the date available or by discovering new combinations of IF-THEN statements that were not originally devised by the programmer. Extrapolation, in brief, means that if an algorithm encounters a situation never seen before, say a Netflix user that has watched The Crown, Glee and Little Baby Bum, it will still give a reasonable recommendation for what this person might like to watch next. As for discovery of new combinations, a Stitchfix algorithm could start learning — somewhat by chance — of new combinations between certain boots, shorts and tops that are starting to sell well together even though they were not formally programmed into the system and no data about this combination of clothe items were available. Differently from extrapolation, discovery requires a feedback loop that informs the algorithm how successful its rules are, i.e., are people buying those outfits.
Main takeaway
ML, AI and BD are based on highly complex technical algorithms. Therefore, separating the truly valuable ones from the rest is also a complex ordeal. To be clear, there is no real substitute to using rigorous statistical methods to assess their value. Even then, it is very possible to be fooled (see How to Lie with Statistics). Aside from builders of algorithms, their managerial users and venture capital investors can take a more practical approach to judging their value. An alternative way for business-trained people is to perform an initial “sniff” test on these algorithmic technologies by looking at IF-THEN statements. To determine whether algorithms are useless, somewhat useful or uniquely valuable, do what you would at a New York City pizza joint. Don’t eat the entire pizza. Just sample a few flavors. And, please, don’t let the vendor decide which ones. If it does not taste good, the entire pizza won’t either.
Thales Teixeira was a professor at the Harvard Business School for 10 years. He is the author of Unlocking the Customer Value Chain, published by Currency, and co-founder of Decoupling.co, a digital disruption advisory firm.
