Why It’s Time to Switch to Time Series Analysis

TL;DR: If you use data to understand and influence humans, check that you’re doing it right and read this paper.

Are you in the business of working with humans?

You trying to understand them? Influence them?

Are you using data for that?

If you answered yes to all of the above, then please check out this wonderful paper. The title of the paper is Time series analysis for psychological research: examining and forecasting change. The authors of the paper are Andrew T. Jebb, Louis Tay, Wei Wang, and Qiming Huang.

If you’re going to study human behavior, you need to take time into account. Time and human behavior are inseparable. As time has passed, scientists have gathered more evidence that this is true.

Psychological processes are inherently time-bound, and it can be argued that no theory is truly time-independent (Zaheer et al., 1999).

The passage of time has also brought us new and wonderful ways to act on this knowledge. The authors explain how a confluence of trends, such as easier access to larger quantities of time-stamped population data, plus tools and techniques in technology and psychology, make it easier than ever to take start taking this approach to data analytics and statistics. Time series analytics provide richer, more accurate views into how our world and its humans behave.

Cause of Death

Unfortunately the use of time series analysis in psychology hasn’t become mainstream. Instead we continue to rely on the tried and true methods of the past. That means collecting smaller amounts of data very close together. We stop collection when we think we’ve reached enough observations to answer a specific question, say 20–50 data points. The entire collection of data is then analyzed and treated as if they all took place at the exact same moment in time. This approach is designed to explain what happened in the past, and why in a limited way.

For some reason this strategy reminds me of a quote attributed to Sir Ronald Fisher:

“To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of.”

Predicting the Future

Compared to the old methods, time series analysis offers everything that batch processing does and much more. We can still look backward to understand why something happened in the past. In addition to that, time series analysis makes it possible to monitor events and trends in real time, to understand what is happening in the present. The best part of time series analysis, is predicting what will happen in the future. Its application to forecasting is why it’s so popular in econometric, financial, and atmospheric studies.

If you think about it, the only reason data is collected and analyzed is to use information about the past to affect the future. That’s why treating data as a function of time is the better approach.

Sadly, data is sometimes collected in a way that doesn’t lend itself to time series analysis. It works fine for analyzing the data in a single point in time, but it’s impossible to take out the noise of seasonal trends or make any predictions. That’s why it’s important to approach data collection with the analysis in mind. Understanding and influencing human behavior starts with good data.

Even though this paper was intended for academics and those in “psychological research” I hope a broader audience will read it, especially if your job involves employee and customer culture. (If you are in the world of startups, marketing, or human resources, and time series analysis is something you do or are thinking about doing in your organization, please let me know! You can reach me on Twitter, acornanalytics.io or the comments below.)

This post attempts to touch on the paper and give you some additional food for thought. Hopefully it gave you just enough of a taste that you’ll read it directly. I hope it didn’t spoil your appetite. Here is the link again:

Jebb AT, Tay L, Wang W, Huang Q. Time series analysis for psychological research: examining and forecasting change. Frontiers in Psychology. 2015;6:727. doi:10.3389/fpsyg.2015.00727.

There will be more blog posts related to this topic, but if you have specific verticals or requests please leave them in the comments.