Debug Keras Models with TensorFlow Debugger
TL;DR
- I want to debug a model created with Keras.
- Keras doesn’t have a built-in debugger.
- Wrap Keras’ Session object with TensorFlow debugger(tfdbg).
Keras, open-source neural networks library, is based on computation graphs. A typical Keras program consists of two separate stages:
- Build the neural network model with Sequential Model or Functional API.
- Training or predicting on the graph.
If errors and bugs occur during the second stage, they are difficult to debug because we don’t know under what situation the computation failed since the running graphs internal structure and state don’t be exposed.
So far, Keras doesn’t have a built-in debugger while TensorFlow have a debugger, called tfdbg. By using tfdbg, we can know the internal structure and state during training or prediction.
In this post, I will introduce how to debug Keras models with tfdbg. It is assumed that Keras backend is TensorFlow since I will use tfdbg. Let’s see tfdbg with a short snippet of code that runs a simple sentiment analysis.
import keras.backend as K
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers import Dense, Embedding, LSTM
from keras.datasets import imdb
from tensorflow.python import debug as tf_debugsess = K.get_session()
sess = tf_debug.LocalCLIDebugWrapperSession(sess)
K.set_session(sess)max_features = 20000
maxlen = 80
batch_size = 32(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
x_train = sequence.pad_sequences(x_train, maxlen=maxlen)
x_test = sequence.pad_sequences(x_test, maxlen=maxlen)model = Sequential()
model.add(Embedding(max_features, 128))
model.add(LSTM(128, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(1, activation='sigmoid'))model.compile(loss='binary_crossentropy',
optimizer='adam', metrics=['accuracy'])
model.fit(x_train, y_train, batch_size=batch_size,
epochs=15, validation_data=(x_test, y_test))
score, acc = model.evaluate(x_test, y_test,
batch_size=batch_size)
In this example, the session object is wrapped as a class for debugging (LocalCLIDebugWrapperSession), so the command-line interface of tfbdg will be launched when the code is launched. Using mouse clicks or commands, you can inspect the nodes and attributes in the graph. Please see the documentation in detail.
Final words
- I want to debug a model created with Keras.
- Keras doesn’t have a built-in debugger.
- Wrap Keras’ Session object with TensorFlow debugger(tfdbg).
