Confining LSTM Train-set Size Based on Batch Size

Caner
1 min readMar 25, 2020

--

I have explained

  • the importance of the batch-size for deep learning models such as LSTM and CNN and also how to select the optimal batch size value here,
  • and here I explained sliding window and timesteps for LSTM models.

Now its time to confine the training set size according to the batch size value e.g. 64. In order to do to train a LSTM model the training set size must a number which is dividable by the batch-size without the remainder or in mathematically;

Computing the maximum train size might be time-consuming at least 5 or 10 minutes. So the function below, I believe that the function below is quite self-explanatory, and might be handy for you.

#df your dataframe
test_percent=0.1 # as an example
batch_size = 64 # as an example
def get_LSTM_train_size(df, batch_size, test_percent):
# substract test_percent to be excluded from training, reserved for testset
number_of_samples = df.shape[0]
print("# Shape of the input dataframe",number_of_samples)
number_of_samples *= 1 - test_percent
train_set_sizes = []
for size in range(int(number_of_samples) - 100,int(number_of_samples)):
mod=size%batch_size
if (mod == 0):
train_set_sizes.append(size)
print(size)
return (max(train_set_sizes))

--

--