FbProphet — Your Solution to Forecasting Problem
We have seen multiple breakthroughs in Natural Language Processing and Computer Vision in the domain of Artificial Intelligence. And we have seen these being applied to many applications to solve language and image-based problems. A common problem crucial to businesses around the globe is Forecasting. This can range from forecasting sales to forecasting footfall. Being able to judge how many people will visit a park or how many soda cans will be sold can be of vital importance in business; to regulate the number of employees needed in the park to cater X number of visitors in a day or the number of items that need to be ordered to manufacture X number of soda cans in a month.
Forecasting can be hourly, daily, weekly, monthly, or yearly, depending on how the business works.
Forecasting is a broad domain and it’s tricky because there are a lot of variations in trajectories of sales of different items or footfall in different locations and it’s harder to take and apply the same parameters to a different problem. And that makes it hectic to use ARIMA or LSTM for forecasting. Even though there’s AUTOARIMA for automatic selection of parameters, ARIMA performs poorly on unstructured time-series data.
In such cases, FbProphet is your savior; easy, fast and gives good performance. If you know how FbProphet works, you can use it well to fit your time-series data. So what is FbProphet made up of:
- Trends
- Seasonalities
- Regressors
Let’s try to understand each of these a little better. Trend is a general trajectory that the data is following or in other words a general direction in which something is developing or changing. Here’s an example of the trend for Peyton Manning’s playoff appearances over the years:
Seasonality is a repetitive pattern the model learns. There could be weekly seasonality; a pattern that generally repeats over every week. Or a monthly pattern that generally repeats itself over a year. Yearly and weekly seasonalities are built-in in FbProphet. But you can also add custom seasonality e.g a pattern that repeats on Saturday & Sunday only. In this case, your seasonality column will have ‘True’ for Saturday and Sunday dates and be ‘False’ for the rest of the days.
Since seasonalities are dependent on Fourier order, you can increase Fourier order to capture higher frequency changes or decrease to learn a more generic pattern.
A regressor adds an additive or multiplicative effect to the model learning. A regressor column must be present in both training and prediction data frames. For example, if we are predicting footfall for a theme park, it’s a good idea to take weather into account. We can add temperature as a regressor. As a result, the model will learn the additional effect of temperature on the number of people that visit the park on a day.
You can also add holidays effect in the forecast. There are two ways of doing this.
- You can specify the country and FbProphet would add holidays for that country.
- If you have other recurring events that you’d like to model, you can create a data frame for them and pass them to the model as holidays, and expect the model to learn and add their effect on the forecast
The model will learn and model the effect of those holidays on the forecast as can be seen below:
When you train a FbProphet model, you can use plot_components to see what trends and seasonalities the model has learned. Below we can see an output for trend, yearly seasonality, and weekly seasonality.
You can adjust the trend flexibility using changepoint_prior_scale. By default, this is 0.05. You can increase it to make the model more flexible to changes (overfit) or decrease it to make it less flexible(underfit). In simpler terms, this parameter can be used to adjust your models behavior between overfitting and underfitting.
CODE
Let’s now dive a little into the code with a footfall dataset for a recreational place in the UK:
import pandas as pd
from fbprophet import Prophetdf = pd.read_csv('./data/footfall.csv')
We’ll rename the columns to ‘ds’ and ‘y’ as required by the FbProphet. We’ll use the built-in weekly and yearly seasonalities and add a custom monthly seasonality as well. The period will be 30.5 (avg of 30 & 31) and we’ll use a Fourier order of 5. We’ll also add country holidays.
m = Prophet(weekly_seasonality=True,yearly_seasonality=True)
m.add_seasonality(name='monthly', period=30.5, fourier_order=5)
m.add_country_holidays(country_name='UK')
m.fit(df[['ds', 'y']])FbProphet will return predictions as yhat with upper and lower uncertainty levels (yhat_upper & yhat_lower).
#predicting for the next 100 days from 2019-04 to 2019-08
future = m.make_future_dataframe(periods=100, include_history=True)
forecast = m.predict(future)
When we keep include_history as True in prediction, the model does an in-sample fit as well (gives predictions for the training dataset as well). You can use this to judge better how well the model has fit both the past and future data.
Let’s do some plotting to better understand the results
fig1 = m.plot(forecast)
fig2 = m.plot_components(forecast)
We can see the footfall count is generally increasing over the years. We can also see from the weekly seasonality plot, that the model has learned correctly a high footfall trend for the weekends.
There are a lot of options available for training a FbProphet model, from regressors to changepoint_prior_scale. More information can be found here.
