Analyst Interrogative #4
How should you define that?
Definitions are often not as easy or straightforward as we expect. I have written before about the difficulties in defining even obvious things. It gets even harder when our definitions must be broadly accepted.
Because of this, asking how something should be defined is one of the most important things an analyst can do. Our definitions will have a lasting impact on the success of our analysis and the businesses that depend upon it.
Scope and Qualification
The online channel has increased the complexity of defining once simple terms like customer. How do you define a customer? Is it someone who pays you for your product or service? What if you have an advertising model based on advertising? What if you have a hybrid model? A freemium model? A transactional model?
Analysts need to think long and hard about the scope and qualifications of anything they want to define. This thought process needs to be informed by basic logic, business need, and the capacity of your systems to collect the data and feedback you will need.
For example — if a company with an advertising revenue model is counting all readers as customers, they will also want to consider the implications of products like AdBloc Plus. If their systems are designed to recognize it, then you might treat that one way. If they are not, it is less clear.
And lest you think this is a purely online issue, consider this. Suppose a certain sister buys her brother a membership to the Tool of the Month Club? Who is the customer? The sister? She may have no more interaction with this service than entering her credit card number every December. The brother is more likely. Maybe it is both? Does your system even acknowledge gift giving?
Average what?
Average is another subject that is taken for granted. Coming from a background in Earth and Environmental Science, I am always amused by pronouncements on global temperatures.
Consider the staggering complexity of determining that average global temperature! Do you really think anyone can measure every cubic millimeter of atmosphere across the globe? At what height do you stop? How do you set the sample frequency — daily, hourly, weekly? How in the world is that measure anything more than a biased estimate? If you read that as a political statement, you missed the point.
Averages that are defined by convenience are likely doing you and your company or discipline a disservice.
This applies to time series tools like moving averages. How do you decide whether to use a 90 day or 6 month window? In certain disciplines, these ‘averages’ are termed ‘smoothing’. They are used to filter out the noise. Yet, may other disciplines employ them for descriptive purposes. Still others for simplification of benchmarks and measurement.
Averages are capable of all of these things. That is the problem. If you are employing it to do one, be wary it is also doing the others. Data that does not adhere to normal distributions is liable to be distorted by the filtering component of an average. Many an insight was lost by the use of a poorly defined average.
How is it going to be used?
It all comes down to your purpose and intent. Analysts need to apply definitions that will support the decisions they are trying to make. Think about the actions that should follow. When you do, focus on your definition and NOT the cute title someone gave it.
The actions you take after learning that your average customers time on site is declining will be very different than after learning that the site time divided by the count of distinct session IPs as collected between January and June is lower than that same ratio between December and July of last year. The latter gets worse if we must also add channels, platforms, sampling frequencies, policy changes, and the like…
How Should I Define That? Maybe The Single Most Important Question For Most Analysts.
Stay tuned for our next installment — Analyst Interrogative #5
Analyst Interrogative is a series created by Corsair’s Institute to increase the reader’s understanding of the analytic process. For more articles like these — please subscribe to Comprehension 360.
