A Case for Ambiguity in Data-Driven Marketing
The field of epistemology is one of the core pillars of philosophy, and it can be paraphrased into a question that gets to the soft and wiggly center of any “truth” we try to defend: “How do you know what you know?” Epistemology is the study of the origins of knowledge, and this question of how we know anything is often glossed over in our daily lives because it is usually more convenient to just accept what we are seeing as true enough to help us determine our courses of action.
“In God we trust, all others bring data” –William Edwards Deming, Statistician, Engineer, and Author
In marketing, we point at data in the forms of customer feedback or online/mobile tracking of users as the source of our knowledge. We trust the numbers. We trust the numbers so much, in fact, that every major digital marketing channel now makes available some form of tracking to link digital events back to ads as a cost-free part of their platform because it is assumed this tracking will be used to justify more ad budget. This technology is a formidable source of insight into the chain of facts surrounding a “conversion,” and we can draw conclusions in the form of receptiveness from audiences or the ability to generate revenue. All the same, the limitations of this data come both in 1) the narrowness of what is reported and 2) the separation between facts and meaning.
Rather than looking at how we do marketing analytics as a “framing” issue by assuming the data can reveal the truth as long as we are identifying the right goals, marketers will find greater freedom to set and adapt smart strategies if they recognize the ambiguity inherent in the data we collect. In other words, marketers should consider marketing as more of an exploration of shifting probabilities rather than the uncovering of the truth about customers.
The epistemic problem in marketing
With sales, the connection between the work a salesperson does and the revenue for which they are responsible is fairly direct. Sales sells to prospects, completes the sale, and reports the revenue. The salesperson has limited data in terms of overall numbers of transactions, but they have their own understanding of the relationships they manage, the specific needs of their clients, and the tactical elements of their sales process that seem to work. Marketers tend to have much broader fields of influence and far more tenuous connections between their efforts and how they shape the growth of a business.
For example, television and radio broadcasts blanket entire regions, helping to raise awareness of products or events for the individuals within those regions. Tracking the efficacy of these campaigns is done via statistical analysis, either at the user level by way of surveys or at the regional level by testing ads in some markets (but not others) and looking for higher revenue gains (or lesser losses) in the test areas as compared to the non-test areas. We start with a hypothesis that the ads will not generate a noticeable outcome and then we test to disprove this hypothesis.
In this example, however, we still cannot remove all doubt that these test regions were responding to other factors, and the overall process of determining the effectiveness of the ads being tested takes enough time that the conditions of the test may not accurately reflect the conditions of the actual competitive environment by the time we commit to an overall launch. Our knowledge is imperfect. We run campaigns anyway because the knowledge we have is often the best we can find among the impossible ambiguity of human interactions. The math is good enough that we know there’s a relationship between the ads and the public’s response, but we do not necessarily know why they respond.
Digital marketers have more tools available to increase the accuracy of our understanding of the customer journey, as well as an incredible amount of collected data from which we can draw potential theories to test. The conflagration of databases, audience pixels, first party data, “onlined” data, third-party data, and data management platforms to parse and organize this information over the past two decades is staggering to the uninitiated. We now have more information about audiences and better insight into digital customer journeys than ever before, which enables more accurate targeting and better personalization of ads, ad experiences, and digital experiences writ large.
The information we glean from all of this structured data allows us to see valuable probabilities (e.g. audience A is more likely than the overall population to be interested in topic D or audience B is more likely to engage in online behavior C), but we still basically have little direct understanding of why the relationships between these audiences and their affinities or behaviors exist. We have ways to demonstrate that we know a conversion happened, but this is not the same as knowing the source of a customer’s motivation to convert.
Why do people see value in [stuff]? How do you know?
The limits of attribution
We talk about conversions and tracking using terms like “causal” to reference the relationship between marketing activities and end results, with the implied conceit that our marketing efforts caused audiences to act rather than some intrinsic motivation tied to the actual utility of the product or service offered. While Coca-Cola may have some credence to such a claim, the vast majority of us do not.
Instead, we have systems of tools that enable us to attribute activities to actions as if they were causally linked the way actions in physics tend to be (e.g. Newton’s third law of motion: every action has an equal and opposite reaction). We can launch a search marketing campaign, connect our ads to a conversion pixel, and get a report for every time someone clicks on an ad and then goes on to do something on a website that “fires” the conversion pixel. We know they saw enough value at the time to convert without knowing why, and this is works fairly well when executed correctly.
When we run multi-ad or multi-channel campaigns, we can have several of these platforms — including search, social, and display networks — doing the same task of delivering ads and reporting on when someone who interacted with an ad does a desirable action on site. Attribution becomes much more difficult when a member of the target audience is served ads on or interacts with ads on multiple platforms before doing that desirable action. Which ad “caused” the action to occur? Each ad platform will claim the conversion as their own, leading to duplication and over-reporting. How do we share credit if multiple actions are to be considered the cause? In this way, even getting a clear understanding of this limited version of causality is less than clear.
An answer to attribution sharing has been to assign weight to interactions as part of this multi-touch, cross-channel customer journey. The most common model for the past decade or so is something called a “last touch” model, which basically just gives credit to the last channel where an ad was clicked. Google Analytics reporting defaults to last click reporting, which makes perfect sense given Google’s historical revenue generator has been Google search ads (generally a down-funnel, final interaction before conversion occurs). Other basic models are available, including models that give credit to the first ad that leads a user to a site, models that distribute credit evenly across multiple touches, and others that spread attribution in other ways.
Generally, a good marketer or analyst will experiment with many models before settling in on one they find instructive for what to do next. All the same, these models are ways of adapting to customer motivations without actually discovering them. At Accomplice, we are developing models that adjust their own weight distribution to enable our automated optimization to work more efficiently. This helps us lead clients farther down the path to understanding but ultimately does not bridge the knowledge gap between the sequence of events we observe and the actual reality of why our target audiences do what they do. It will work better for doing marketing, but the tool we offer is still just a machine.
This question for marketers — “how do we know” that our marketing is effective — is a critical question because we never truly know what motivates purchaser behavior, even though the technology we now have available to use makes it possible to test faster, refine our theories, optimize our outcomes, and narrow down the scope of what could possibly lead a prospect to become a customer. Unlike some philosophical takes on the epistemic question, where we reach an impasse of infinite regression (like a toddler asking “why” after each successive explanation), we have the benefit of falling back on fallibility, testability, and the realization that ambiguity is not so terrible after all. There’s a good reason marketing is considered both an art and a science.
Schrödinger’s marketing campaign (recognizing ambiguity)
Though we have addressed the kind of physics many learn in high school or their freshman year of college, more contemporary physics embraces a much less mechanistic understanding of the universe. In quantum mechanics, physicists embrace ambiguity and uncertainty in the way our universe manifests itself. The basic particles that make up all of the things in existence behave strangely at the infinitesimally small scale. One of the more famous observations of these particles is the Heisenberg uncertainty principle: the more precisely we observe the position of a particle, the less precisely its velocity can be observed. Otherwise stated, we can know where a particle is or how fast it is moving, but not both things at the same time. While our consciousness of marketing does not work precisely the same way, there are some examples of Heisenberg’s principle that we can use by analogy to help us understand how ambiguity in marketing can work for us.
Among the more common ways to look at the Heisenberg uncertainty principle is a thought experiment devised by Erwin Schrödinger concerning a cat, a vial of poison, and a source of radioactivity. The experiment posits a scenario where the cat is locked in a box with the poison and the radioactive material, and if a monitor inside the box detects radiation (from particle decay) the poison is released and the cat dies. Until the box is opened, and the cat is observed, the cat exists to viewers of this horrible spectacle as if it is present in all of the possible states available to it: alive and dead.
Until we are measuring a marketing campaign, it exists in a similar sort of multi-possibility state, but by measuring the campaign we collapse the overall number of probable outcomes of its performance down to what we have observed. We make decisions based on what we see and test for other possibilities from our campaign, revisiting and redirecting the overall field of possibilities available to our campaigns until our time or budgets run out. At Accomplice, this process of repetition is part of how we automate optimizations, acknowledging the simple fact that every time we make a decision we open up the field of possibilities to test again and learn anew what generates the kind of performance we are seeking.
Every time we launch a campaign, we are embracing ambiguity, if only until we can crunch the numbers to determine our performance and set new campaign parameters. Likewise, even if we have an understanding of current campaign reporting, we do not have access to the evolving motivations of clients or the great aggregate of other factors that appear by chance to skew our results. Where Heisenberg’s principle proves to be true regardless the reach of our technology, it is not yet clear whether marketing will be unable to bridge the gap between performance and understanding.
Pursuing understanding from the limited capacity of the knowledge we can develop from technology gets the roles of marketing and technology fundamentally backwards. If we build our campaigns around theories regarding the motivations of our audiences, the technology allows us to quickly disprove erroneous and irrelevant theories and begin forming harder to disprove understandings of audience motivations. Marketing, like economics and sociology, is a study of motivations and cultural factors that contribute to social interactions. Framing the use of data, or a test for motivations for that matter, is important to marketing success, but it is only a smaller part of the greater realization that what we are doing is leveraging technology to learn more about why people make decisions and take action.
Until we are studying motivations, even as we frame our tests in the context of generating the revenue that keeps our companies or clients afloat, we are condemned to do our work amidst the uncertainty of an infinite array of possibilities for why our campaigns succeed or fail. The technology makes it easier to forget how little we actually know.
Addressing the knowledge gap
The framing of many marketing campaigns starts with a strategy in the form of a story: Here is a product or service that has a particular value, and it will be valuable to some segment of the population, and this segment will be approached with messaging that will appeal to them via channels they tend to use. If we come from the context of the management theory of Peter Drucker, we know how critical getting this story right really is in terms of our responsibility to our clients and companies.
“Because the purpose of business is to create a customer, the business enterprise has two–and only two–basic functions: marketing and innovation. Marketing and innovation produce results; all the rest are costs. Marketing is the distinguishing, unique function of the business.” — Peter Drucker, Management Consultant, Educator, and Author
The difference between a target audience member and a customer can be fairly stated as a conversion, which we often see in marketing. This is not just jargon because the people we market to go from one state of being (target) to another (customer). Unless we can observe the marketing we are doing, these targets or customers effectively exist in both states. The measuring we do as part of our work as marketers needs to be geared toward testing theories regarding why people become customers. Drucker’s emphasis on innovation is not accidental, as innovation is the development of new utility or value for potential customers, and the work of marketing is to determine and execute the best course of communicating this value in a way that resonates with a customer’s motivations. We do more than just drink coffee and wave our hands.
For a salesperson, exploring motivations in a one-to-one setting is part of a conversation. While a savvy salesperson may intuit or extrapolate in order to identify other motivations, sales is still a function of matching up the final details connecting customer motivations to expected product value. Marketers are tasked with defining the entire context of a sales conversation or create the opportunity for a customer to simply convert on their own. Acknowledging that we do not know why a customer converts, rather than simply asserting the value we think is intrinsic in our product or service as a proxy for that motivation, starts us on an honest path of asking questions about how to close the gap between what we can measure and what we can understand.
Asking customers why they chose to convert is as important now as it was before the proliferation of digital marketing technology and channels. Soliciting product or service feedback and listening to how customers and prospects describe their motivations is as important now as it was before the proliferation of digital marketing technology and channels. Stimulating discussions with customers regarding how we communicate new products, services, and features is as important now as it was before the proliferation of digital marketing technology and channels.
This likely seems basic, especially given the references to physics and philosophy, but these are fundamental actions for marketers who intend to understand their customer and run successful marketing programs. Marketing data and marketing measurement are incredibly powerful tools that refine down the minutiae of campaign effectiveness, but the over-dependence on these forms of knowledge can lead to data-driven myopia regarding the bigger picture of what marketing is meant to do.
We communicate value.
Everything else we do must be in service of this function, and the testing we do needs to be centered around the search for probable motivations rather than simply maximizing KPIs. The technology we use helps us to better understand whether we are communicating effectively and should be used to help us test the messages we create to address probable customer motivations. Realizing a KPI maximum for a single campaign will not drive the kind of marketing success a working understanding of customer motivations will bring in the long run.
Conclusions
If we begin with the assumption that our knowledge and understanding are fallible, and we interact with customers directly or via our marketing campaigns with the goal of invalidating all of our theories about what motivates conversions, then we have a hope of narrowing down what motivates customers to convert. With that knowledge, we can re-form marketing efforts and concentrate product development in useful ways for our clients and company performance.
We cannot be afraid to be wrong because we need to be wrong often enough to understand how to do our jobs right. In the end, we still will not completely bridge the gap and fully understand the depth and complexity behind the motivations of customers. We will, however, be able to answer for all of the motivations we know have not lead to conversions. More importantly, we will be able to answer for all of the customer motivations we can continue to use in tests because we have not yet been able to invalidate them using the technology we have available. That’s far more useful than the limited number of things we can know for certain.
Our campaigns are experiments to figure out how to better serve our customers and their motivations. If this is our goal, the money will follow.
Many thanks to Idan B and Jess W for reviews and edits prior to the publishing of this version of the post.
If you liked this post, please do me the favor of recommending it and/or sharing with your friends and colleagues through social. I wrote this to distill and share some of what I’ve learned with my coworkers, clients, and readers. If it’s something you like then I’d be honored for you to spread this post as well.
As well, this post now appears on the Accomplice company blog. Feel free to look there for it.
Also, feel free to reach out of me on Twitter (@indasein) for any comments, feedback, or to share your own thoughts. Thanks!
