Website visitor forecast with Facebook Prophet: A Complete Tutorial
With installation instructions, data, parameters tuning, and both simple & sophisticated forecasting— Part 2
This is part 2 of the tutorial — Website visitor forecast with Facebook Prophet all in one tutorial 2022 (with installation instructions, data, parameters tuning, and both simple & sophisticated forecasting).
The original article covers,
- Installation of Prophet
- Simple ETL, and Data Visualisation
- Simple forecasting (forecast with default settings)
- Forecasting with model assessment
- Forecasting with tuned parameters
However, the original article was too long. Therefore it is now divided into 2 — the first 3 sessions are in part 1, while session 4 and 5 are in part 2. If you are not interested in installing and simple forecasting, please click here to part 2.
4. Forecasting with Modal Assessment
Assessing the model accuracy is essential for creating machine learning models to describe how well the model is performing in its predictions. Fortunately, we can employ some common yet easy evaluation metrics when assessing our Prophet’s performance in prediction.
* MAE (Mean absolute error)
- averaging the absolute difference between the original (y) and predicted values (yhat) over the data set. The smaller, the better.
* MSE (Mean Squared Error)
- averaging the squared difference between the original (y) and predicted values (yhat) over the data set. This is good for datasets with outliners, which will be penalised heavily with this metric. Again, the smaller this value, the better the model.
* RMSE (Root Mean Squared Error)
- is just taking the square root of MSE.
* R-squared (Coefficient of determination)
- a coefficient of how well the model is explaining the object being studied. It values from 0 to 1. The bigger, the better.
def evaluate(y_true, y_pred):dict = {‘MAE’: mean_absolute_error(y_true, y_pred),‘MSE’: mean_squared_error(y_true, y_pred),‘RMSE’: sqrt(mean_squared_error(y_true, y_pred)),‘R2’: r2_score(y_true, y_pred)}return dict
In our analysis, we will focus on MAE and R-Squared only. But I will still include the MSE and RMSE in the code.
I will first divide the dataset into 2, one for training and testing.
test_len_assess = 8train_df_assess = train_test_df_assess.iloc[:-test_len_assess, :]test_df_assess = train_test_df_assess.iloc[int(-test_len_assess):, :]
The number of 8 is not random. Some readers have asked before, why number 8? It is the last 10% of the dataset divided by 2. It is to match the dataset in the tuning part of this tutorial.
Let’s predict and evaluate our model now.
m = Prophet(interval_width=0.95, weekly_seasonality=False, daily_seasonality=False)m.add_country_holidays(country_name=’DE’)
m.fit(train_df_assess)predict_df_assess = m.predict(test_df_assess)evaluate(test_df_assess[‘y’], predict_df_assess[‘yhat’])
There are many columns in the predict_df_assess dataframe. They all are meaningful but out of the scope of this tutorial. I will explain more if I have another article about it. Please follow me on Medium to stay informed with my latest data science articles like this. Your following and Like is a great motivation to keep me writing.
In this tutorial, we will only focus on the yhat in the last column of the data frame. Prophet’s model with default parameters always looks good without a scientific evaluation. Now we can see it has 60% of explanation power.
Visualize our result and the forecast portion.
plot_measure = m.plot(predict_df_assess)
Let’s predict the future with this evaluated model.
future_assess_input = m.make_future_dataframe(periods=60,freq=’W’)future_assess_output = m.predict(future_assess_input)plot_assess = m.plot(future_assess_output)
You can see this model has eight fewer rows when being trained; this is why the forecast is slightly different from our simple forecasting model in the previous session, which has eight more rows.
5. Optimizing with Optuna
Optuna is an automatic hyperparameter optimisation software framework, mainly designed for machine learning. The user of Optuna can dynamically construct the search spaces for the hyperparameters easily.
I am still no expert in this library. To find out more, please click here:(https://optuna.readthedocs.io/en/stable/index.html)
Before we dive into parameter tuning, let’s have a deeper look at the parameters first.
The Prophet model has several input parameters that one might consider tuning. The parameters are slightly overwhelming to start with. However, blind tuning is not ideal. Therefore, understanding the parameters beforehand is needed. Here are some general recommendations for hyperparameter tuning that may be a good starting place.
- changepoint
Changepoint is the point where the trend change. By default, n_changepoints, which stands for ‘number of changepoints’, has 25 change points. The changepoint_prior_scale controls the magnitude of the change. If you find the trend changes are overfitting (too much flexibility), you can increase it; or decrease it if it is underfitting (not enough flexibility). The default is 0.05.
Usually, you are not advised to tune n_changepoints. In my experience, it will likely be more effective to adjust seasonality_prior_scale.
- changepoint_range
It basically is how much data will be taken into consideration when setting change_point by Prophet. The default is 0.8, which means changepoints are only inferred for the first 80% of the time series to have plenty of runway for projecting the trend forward and avoiding overfitting fluctuations at the end of the time series. My experience is that numbers around 0.8, e.g. 0.82, 0.70, will do the job well.
- growth
This is a very tricky param. Options are ‘linear’ and ‘logistic’. I suggest changing this to logistic only if you know your data will saturate within your forecasting period. If it is not, keep it simple and stay linear. Because Prophet will try its best to move your future forecast toward this saturation point in the included period.
If you are very clear that your data is reaching this saturation point, e.g. you are sure you will have 1000 visitors to your website in the coming months, you can choose logistic here and include the ‘cap’ and ‘floor’ values in your data frame.
- seasonality_mode
Options are [‘additive’, ‘multiplicative’]. Default is ‘additive’, but many business time series will have multiplicative seasonality. This is best identified by looking at the time series and seeing if the magnitude of seasonal fluctuations grows with the magnitude of the time series. My experience is that the default additive fits my need most of the time.
- seasonality_prior_scale
This parameter controls the flexibility of the seasonality. Similarly, a significant value allows the seasonality to fit large fluctuations; a small value shrinks the magnitude of the seasonality. The default is 10. A reasonable range for tuning it would probably be [0.01, 10].
- holidays_prior_scale
This controls flexibility to fit holiday effects. Similar to seasonality_prior_scale, it defaults to 10. A reasonable range for tuning it would probably be [0.01, 10].
- yearly_seasonality
By default, it is ‘auto’. ‘True’ will turn yearly seasonality on, and ‘False’ otherwise. Options are [‘auto’, True, False].
As we have weekly data and don’t have daily or hourly data, we won't apply weekly_seasonality and daily_seasonality params.
As aforementioned, we will separate a data frame, particularly for validation, from the primary dataset.
test_len_tun = int(train_test_df_tun.shape[0] / 10) # take only 1/10 as the test sizetrain_df_tun = train_test_df_tun.iloc[:-test_len_tun, :]val_df_tun = train_test_df_tun.iloc[-test_len_tun:int(-test_len_tun/2), :] # within the test pool, the first half is taken for validationtest_df_tun = train_test_df_tun.iloc[int(-test_len_tun/2):, :] # only the final half of the test pool is for the test
Then we create a function called find_params to find the best parameters with the Optuna library.
def find_params(trial):parameters = {‘changepoint_prior_scale’: trial.suggest_float(‘changepoint_prior_scale’, 0.005, 5),‘changepoint_range’: trial.suggest_float(‘changepoint_range’, 0.1, 0.9),‘seasonality_mode’: trial.suggest_categorical(‘seasonality_mode’, [‘multiplicative’, ‘additive’]),‘seasonality_prior_scale’: trial.suggest_float(‘seasonality_prior_scale’, 0.1, 10),‘yearly_seasonality’: trial.suggest_int(‘yearly_seasonality’, 1, 50),‘holidays_prior_scale’: trial.suggest_float(‘holidays_prior_scale’, 0.1, 10)}m = Prophet(**parameters, # ** means unpackinterval_width=0.95,weekly_seasonality=False,daily_seasonality=False)m.add_country_holidays(country_name=’DE’)m.fit(train_df_tun)validation = m.predict(val_df_tun)mae_for_optuna = mean_absolute_error(val_df_tun[‘y’], validation[‘yhat’])return mae_for_optuna
Then we apply the function by study.best_params. We will specify the number of trials in this step. I would recommend 1000 at least. In our Jupyter notebook, you will find my some other trials, including some with only 500 trials.
study = optuna.create_study(direction=’minimize’)study.optimize(find_params, n_trials=1000) #1000study.best_params
You can find the individual result in the printout during the search. The circled number below is the trial number. The first underlined is the MAE of the trial, the lower, the better. And the last underlined is the best result stored in the processes.
After 2–3 hours of waiting time, the search is done, and the result is as below.
para = {‘changepoint_prior_scale’: 1.9804273036896098,‘changepoint_range’: 0.6543491388579227,‘seasonality_mode’: ‘multiplicative’,‘seasonality_prior_scale’: 4.465868155817663,‘yearly_seasonality’: 18,‘holidays_prior_scale’: 2.650571507054187}
We now train our model with the best-found parameters again with a matched length data frame.
train_df_tun2 = pd.concat([train_df_tun, val_df_tun])m = Prophet(**para,interval_width=0.95,weekly_seasonality=False,daily_seasonality=False)m.add_country_holidays(country_name=’DE’)m.fit(train_df_tun2)# Then we test our newly trained model with the test df.predict_df_tun = m.predict(test_df_tun)evaluate(test_df_tun[‘y’], predict_df_tun[‘yhat’])
{‘MAE’: 24025.085572669836,
‘MSE’: 789692022.1894112,
‘RMSE’: 28101.459431663174,
‘R2’: 0.7987997251205757}
R² shows that our newly trained Prophet model has 80% explanation power. Not bad at all! It is much better than the 60% of the previous untuned model. There is an impressive 32% model improvement.
let’s create our final dataframe and visualize it.
future_optuna_df = m.make_future_dataframe(periods=60,freq=’W’)predict_optuna2 = m.predict(future_optuna_df)predict_optuna2.columnsforecast_final = predict_optuna2[[‘ds’, ‘trend’,’yhat’]]forecast_fig_final = px.line(forecast_final, x=”ds”, y=”yhat”, title=’The number of unique visitors of www.lorentzyeung.com forecast one year into the future.’)fig_final.show()
Conclusion
Congratulations! You have just learned how to predict with Prophet with out-of-the-box setting, and with optimisation. By parameter specification, one can improve the model substantially without overfitting. In our case, we improved our model by 32%. It could go higher but i will stop for now.
Thank you for reading. If you like this tutorial, please share to your data science friends, and follow me. This following are the motivation for me to continue contributing to the community.
