Time series analysis made easy using Shiny
Introducing TIM: the Time series Insights Maker.
Meet TIM, a new and completely free tool built in Shiny to generate insights from your time series data.
Whether it’s daily sales, weekly traffic or monthly website visits, TIM can help you quickly generate a forecast, find outliers, and identify an event’s impact.
With TIM, you can do things like:
- Forecast sales, customer counts, expenditure, traffic, library visits, or anything else along those lines[1]. For example, a small business owner might like to know what sales may look like for the next few weeks.
- Find periods where your data was higher or lower than expected. A traffic planner could use TIM to find out where and when traffic was unusually high or low.
- Identify whether and to what extent an event positively or negatively affected your data. Analysts can use this to find out whether a COVID-lockdown had an impact on park visitation, a major tourism event increased expenditure in the entertainment sector, or to what extent a marketing campaign impacted website visitations.
Using TIM
Using TIM is straightforward:
- Upload a time series dataset: Upload your own data in csv, xls or xlsx format, or use one of the files that I’ve uploaded in the ‘Example files’ menu if you want to just explore.
- Confirm date and value columns: TIM guesstimates what the date and value columns are, so make sure they have been identified correctly.
- Select analysis type: Select whether you want to forecast, identify outliers, or assess an event’s impact
(a) If you’re forecasting, also indicate how many time periods you want to forecast.
(b) For event impact analysis, select the event’s start and end dates. - Run analysis: Press the button and get your result!
Uploading data
Using your own data
TIM can analyse daily, weekly, monthly, quarterly and yearly data. As long as your dataset contains a column called ‘date’, ‘month’, ‘week ending’ or similar, and a column with data (like sales or traffic), you should be able to plug and play.
TIM does not save any data you upload or results you generate, and as soon as you exit the page your data and results will have disappeared
If you’re unsure whether your data will work in TIM, see if it matches one of the files in the “Example files” menu. Using the Google mobility dataset for the City of Melbourne, you can see that TIM works well when:
- The date column is in day-month-year format. Year-month-day or year-quarter (e.g. 2022 Q1) and other common formats work equally well.
- Each date has one observation. For example, for 15/02/2020 there is only one row of data, with each column being a single variable.
Using the pre-loaded datasets
I’ve uploaded datasets that you can use to explore TIM. Simply go to the ‘Example files’ menu in the sidebar, choose a file from the drop-down menu, and click ‘Download data’. All datasets are described on this page, where you’ll find descriptions and sources.
Example uses
Forecasting
I’m using the weekly traffic collisions In LA dataset, available in the ‘Example files’ menu, to forecast collisions for 12 weeks.
You can see the steps I took in the image below to get there, which were as simple as uploading the datasets, clicking on ‘Forecast’ in step 4 and entering 12 weeks.
In the top left of the image below you can see six key stats that show you TIM’s accuracy — how close TIM’s estimate for collisions is to the actual values in the period before the forecast starts — and a summary of the forecast period itself.
The graph in the top right shows the 12-week forecast period, with the forecast shown in red. The confidence interval is shown using a grey ribbon, which tells you, with 90% confidence, what the range of values is that the forecast will lie between.
The “Full results” box shows the original data (in blue) and the forecast (in red), both showing the model’s confidence interval (in grey). Notice that the forecast picks up the dip that seems to occur during the Christmas period, and seems to generally align with what you might expect would happen.
In the “Full results” box, you can also find the forecast results in table format in the “Detail” tab. Plus, you can download them using the icon on the right hand side. In the “Trends” tab you can look at the underlying trend in your data (after removing all seasonal effects) and, in the case of weekly and monthly data, what the median monthly value is across the year.
Outlier analysis
For the outlier analysis example, I’ve taken the Monthly Water Production in QLD dataset from the ‘Example files’ menu.
After uploading the data, I did not have to change anything in Steps 2, 3 and 4 so I clicked on the ‘Run analysis’ button.
TIM again shows six key stats in the top left. Apart from the model’s accuracy (how close TIM’s estimate gets to the actual values), they show how many days were different from what you might have expected (as in, outside the confidence interval — the estimate of the likely range of forecast values), and which days saw the highest and lowest values across the board.
In the top right, you’ll be able to quickly see which three periods (in this case, months) were the top outliers; the green and red boxes show you which months were the top three that were above (below) the confidence interval.
The bottom half of the page shows you the whole period (in blue) the model’s confidence interval (in grey), and where the data was above (in green) or below (in red) the confidence interval. In other words, the points where your data may have been affected by unique events.
As with the forecast, you can see the details of the outliers in the “Detail” tab, and you can download the results by clicking on the icon in the top right of the box. The “Trends” tab shows you the same information as described in the forecast analysis.
Event impact analysis
To show how TIM’s event impact analysis function works, I’m using the Melbourne Pedestrian Counts daily dataset, available from the ‘Example files’ menu.
After uploading the data, I chose the Bourke Street Mall North sensor in Step 3 (representing a key shopping area in the CBD), select ‘Event impact analysis’ in Step 4, and provided dates where I think an event took place that impacted pedestrian counts. In my case, I suspected that a COVID lockdown in mid to late July 2021 had a major impact on activity in the area, so I entered 15 to 27 July 2021 and click on ‘Run the analysis’.
In the top left, you can again see how accurate TIM is up to the event when comparing its estimate to the actual values, as well as some event impact specific summary information. In this case, all 12 days of the event were lower than expected; it looks like pedestrian counts over the event period may have been 74% lower than the ‘no event’ forecast scenario. The graph in the top right shows the cumulative impact of the event.
In the bottom half you can see the full period, though here, I’ve opted to show only the last six months of data plus the event dates. The event starts at the dotted line, after which the forecast for the event (in red) is shown along with what actually happened during the event (in blue). Like with the outlier analysis, you’ll see green (red) circles if a value is above (below) the forecast’s confidence interval. In this case, the forecast continues merrily along the lines of the period prior to the event, while COVID had a major negative impact on pedestrian counts during that period — explaining the red circles for all event dates.
Using the buttons directly above the graph you can control how much of your data you see.
Once again, you can use the ‘Detail’ tab to inspect the event period’s results, and you can download the data using the icon to the top right of the box. In the “Trends” tab, you will again find the underlying trend in your data, as well as the chart shown below. For daily data, this chart summarises the median value (for pedestrian traffic in this case) by each weekday and month. In this case, pedestrian traffic tends to be highest on Saturdays in December, and lowest on Sundays in February.
How does TIM work?
The forecast model
The modelling technique used in TIM is based on the prophet package. Full documentation can be found here.
COVID
TIM adds yearly seasonality to the data if less than half of all observations in the data were during COVID. The COVID period is defined as the ‘peak lockdown’ period for Australia: April 2020 to July 2021 (inclusive). I found this performs equally well as building custom seasonalities, which Meta suggests as an option, but more straightforward to implement.
Default parameters
Holidays
By default, the model will include New Year’s Day, Easter (Friday to Monday), Christmas, Boxing Day and New Year’s Eve. Any effect happening on these days will be assumed to be a characteristic of the data provided.
Trend flexibility
Extensive hyperparameter tuning using test data indicates that a flexibility (changepoint.prior.scale) of 0.45 generates generalisable trend changes that don’t overfit the data.
Improvements and updates
TIM is new to the world and I’ll be looking to do regular updates. If you have tips, questions, feedback, complaints, compliments, other datasets you think would be good to see in TIM, or anything else you’d like to say, please get in touch!
Footnotes
[1] TIM works well for most time series datasets, but not for data that is highly volatile (like the stock market, cryptocurrency and the like), and/or impacted by a large range of factors (such as interest rates or house prices).
