# Multi-channel CNN for text

Here is an implementation of the paper by Kim et al. for sentence classification using multi-channel CNNs.

I won’t go into details of the text input/preprocessing steps etc. since they are standard pipelines already available on several blogs, as well as Keras’ examples. [Check this post]

The multi-channel architecture constructs 2 channels of word embeddings using Glove/Word2Vec for the input sentences.

On one channel, pre-trained vectors with static weights are used, while in the other, pre-trained vectors with learnable weights are used. It would be interesting to see if having Glove & Word2Vec vector embeddings as the two channels improves the model.

These two channels are fed to a CNN model.

Here’s a simple model that does this:

embed1 = Sequential()

embed1.add(Embedding(len(word_index) + 1,

EMBED_DIM,

weights=[embedding_matrix],

input_length=MAX_SEQUENCE_LENGTH,

trainable=False))

embed2 = Sequential()

embed2.add(Embedding(len(self.word_index) + 1,

EMBED_DIM,

weights=[embedding_matrix],

input_length=MAX_SEQUENCE_LENGTH,

trainable=True))

model = Sequential()

model.add(Merge([embed1 ,embed2], mode='concat', concat_axis=-1))

model.add(Reshape((2, MAX_SEQUENCE_LENGTH, EMBED_DIM)))

model.add(Convolution2D(64, 5, EMBED_DIM, activation="relu", border_mode='valid'))

model.add(MaxPooling2D((MAX_SEQUENCE_LENGTH-5+1,1)))

model.add(Flatten())

model.add(Dense(256, activation="relu"))

model.add(Dropout(0.3))

model.add(Dense(len(nb_labels), activation="softmax"))

model.compile(loss='categorical_crossentropy',

optimizer='rmsprop',

metrics=['acc'])