Time Series Forecasting using LSTM | Code walkthrough

Basil K Jose

Published in

Apes AI

7 min readFeb 3, 2021

Introduction
Why lstm?
Prerequisites
Dataset
Univariate forecasting
Multivariate forecasting
References

Introduction

Time series analysis can be useful to see how a given asset, sensor value,security, or economic variable changes over time. It can also be used to examine how the changes associated with the chosen data point compare to shifts in other variables over the same time period.Today most companies use time series forecasting for business growth so they can plan accordingly in the future.It also used in many other fields ,we can use time series forecasting where we have timely recorded data is available. So it is used almost everywhere we need to predict future outcomes from timely recorded historical data.

Here are a few examples of how different industries use time series forecasting:

Energy — Prices,demand, production schedules,temperature prediction
Retail — Sales,consumer demand for certain products
State government — Sales tax receipts
Transportation — Demand for future travel
Finance — Stocks, market potential

Today time series forecasting is one of the hot fields in data science. Different modeling strategies are used for forecasting we can use statistical, machine learning and deep learning models.In this blog, we focused on deep learning LSTM models. Most of the code snippets used for this blog is taken from the TensorFlow 2.0 documentation. The main aim of this writing step by step code walkthrough from data preprocessing to modeling.

Why lstm ?

Long Short-Term Memory (LSTM) is a type of recurrent neural network that can learn the order dependence between items in a sequence.LSTM are pretty good at extracting patterns in input feature space, where the input data spans over long sequences. Given the gated architecture of LSTM that has this ability to manipulate its memory state(memorizing historical data), they are ideal for time series problems.

Prerequisites

Basic knowledge of LSTMs and RNNs.
Should familiar with Tensorflow 2.0,matplotlib,numpy and pandas.
complete code is written on python and run on the google colab platform.

Dataset

This tutorial uses a weather time series dataset recorded by the Max Planck Institute for Biogeochemistry.

This dataset contains 14 different features such as air temperature, atmospheric pressure, and humidity. These were collected every 10 minutes, beginning in 2003. For efficiency, you will use only the data collected between 2009 and 2016. Our aim in this code walkthrough is to forecast air temperature for future timestamps.

tf.keras.utils.get_file() method in tensorflow is used to download a file from a URL if it is not already in the cache. File is downloaded as a zip file since extract is equal to true, the file is extracted and saved as a csv file.

os.path.splitext() method in Python is used to split the path name into a pair root and ext. Here, ext stands for extension and has the extension portion of the specified path while root is everything except ext part.

Forecasting task: Predict temperature (in deg C) in the future.

Univariate Forecasting

Only one variable is varying over time. For example, data collected from a sensor measuring the temperature of a room every second. Therefore, each second, you will only have a one-dimensional value, which is the temperature.

# univariate dataframe: Temp vs Time
uni_data_df = df['T (degC)']# variable
uni_data_df.index = df['Date Time']#index
uni_data_df.head()

For univariate time series forecasting we want to consider only one variable ,since the problem statement is to forecast temperature.We want the dataframe to consist of temperature as variable and date(timestamp) as index.

Before reaching forecasting, we must understand how important preprocessing is for time series. It can make or break your forecasting. So let us go through some of the crucial preprocessing steps for time series.The main challenge to forecasting is preprocessing data into the relevant form.

TRAIN_SPLIT = 300000 # First 300000 obs will be used as train data and rest as test data.
tf.random.set_seed(13) # random seeduni_train_mean = uni_data[:TRAIN_SPLIT].mean()
uni_train_std = uni_data[:TRAIN_SPLIT].std()uni_data = (uni_data-uni_train_mean)/uni_train_std

In time series forecasting we cannot do the random splitting of data. Because we want to keep all the data in order of timestamps, so in this problem, we considered the first 30000 data points as training data and the remaining as cross-validation data.The reason for standardization is that the model runs very fast and also to bring values to specific ranges. Using train data we standardize our entire dataframe

def univariate_data(dataset, start_index, end_index, history_size, target_size):
    """function to create input and ouput values for univariate time series forecasting"""
  data = []
  labels = []

  start_index = start_index + history_size
  if end_index is None:
    end_index = len(dataset) - target_size

  for i in range(start_index, end_index):
    indices = range(i-history_size, i)
    # Reshape data from (history_size,) to (history_size, 1)
    data.append(np.reshape(dataset[indices], (history_size, 1)))
    labels.append(dataset[i+target_size])
  return np.array(data), np.array(labels)

# use the above function to create the datasets.
univariate_past_history = 20
univariate_future_target = 0x_train_uni, y_train_uni = univariate_data(uni_data, 0, TRAIN_SPLIT,
                                           univariate_past_history,
                                           univariate_future_target)
x_val_uni, y_val_uni = univariate_data(uni_data, TRAIN_SPLIT, None,
                                       univariate_past_history,
                                       univariate_future_target)

We already said that one of the challenging parts of time series forecasting is converting data into input and output format. The above function is used for this purpose.

In this problem we considered univariate_past_history = 20 which means we are considering past 20 records for prediction of the next value that is the first 20 values of the temperature variable is x1(indices 0 to 19) and x2 will be indices from 1 to 20. univariate_future_target = 0 means we want to predict the value at the current index that means y1 will be indexed at 19 and y2 index at 20.If univariate_future_target = 5 then y1 is index at 24.(using previous 20 timestamps record want to predict 25th timestamp temperature)

# TF Dataset preperation
BATCH_SIZE = 256 # bacth size in batch-SGD/variants
BUFFER_SIZE = 10000 # for shuffling the datasettrain_univariate = tf.data.Dataset.from_tensor_slices((x_train_uni, y_train_uni))
train_univariate = train_univariate.cache().shuffle(BUFFER_SIZE).batch(BATCH_SIZE).repeat()val_univariate = tf.data.Dataset.from_tensor_slices((x_val_uni, y_val_uni))
val_univariate = val_univariate.batch(BATCH_SIZE).repeat()

A batch size of 256 is fed in each epoch, using the shuffle function data points is shuffled across each batch for an indefinite time using the repeat function.

# MODEL:
simple_lstm_model = tf.keras.models.Sequential([
    tf.keras.layers.LSTM(8, input_shape=x_train_uni.shape[-2:]),
    tf.keras.layers.Dense(1)
])simple_lstm_model.compile(optimizer='adam', loss='mae')
# Train and evaluate
STEPS_PER_EPOCH = 200
EPOCHS = 10simple_lstm_model.fit(train_univariate, epochs=EPOCHS,
                      steps_per_epoch=STEPS_PER_EPOCH,
                      validation_data=val_univariate,  
                      validation_steps=50)

Here we used a simple lstm model for forecasting consisting of one layer.MAE is used for loss minimization , hyperparameter tuning is to be done on modeling.

Multivariate Forecasting

A Multivariate time series has more than one time-dependent variable. Each variable depends not only on its past values but also has some dependency on other variables. This dependency is used for forecasting future values.Mainly there are two types of multivariate time series forecasting.

Multivariate single step forecasting
Multivariate multiple step forecasting

For multivariate time series forecasting we considered three features

Using train data we standardize our entire dataframe .

def multivariate_data(dataset, target, start_index, end_index, history_size,target_size, step, single_step=False):
  data = []
  labels = []start_index = start_index + history_size
  if end_index is None:
    end_index = len(dataset) - target_sizefor i in range(start_index, end_index):
    indices = range(i-history_size, i, step) # step used here.
    data.append(dataset[indices])if single_step: # single_step used here.
      labels.append(target[i+target_size]) 
    else:
      labels.append(target[i:i+target_size])return np.array(data), np.array(labels)

There is a slight change in function to create input and output values. we have added two new parameters to the function.

step: instead of taking data for each timestamp, do you want to generate data once every n steps . (for example if step = 6 ,then function consider data on every sixth time stamp)

single_step: labels(output) from single timestamp or multiple time steps. It takes either True or False. If it is true then time series forecasting is Multivariate single step forecasting , that is we are predicting for only one timestamp value in the future. If the value is false then time series forecasting is Multivariate multi step forecasting, in future predicting multiple values at different time stamps

All other steps like tensor dataset creation, modeling remains the same as univariate time series forecasting.Entire code is present in github.

You can find my complete code in my Github Repository ,and if you have any suggestions, please contact me via Linkedin

References

https://www.tensorflow.org/tutorials/structured_data/time_series#%20Reasoning%20for%20explaining%20TF%20guides/tutorials:
https://machinelearningmastery.com/deep-learning-for-time-series-forecasting/