Handwritten Digit Recognition Using Neural Network
“The MNIST database of handwritten digits, available from the website, has a training set of 60,000 examples, and a test set of 10,000 examples.” Each example is an 28x28 image of a handwritten digit.
這個範例綜合了先前所介紹的Neural Network, Stochastic Gradient Descent 和Backpropagation。
首先，先建立一個三層的neural network，架構是第一層input layer（共有784個neurons），第二層hidden layer（共有20個neurons），最後一層output layer（共有10個neurons），並將weights和bias賦予起始值，如下圖。
# build the network
# w1/b1 w2/b2
#784(inputs) ---> 20 ---> 10(output)
# x z1 a1 z2 y
self.weight1 = np.random.normal(0, 1, [self.num_nodes_in_layers[0],
self.num_nodes_in_layers[1]])
self.bias1 = np.zeros((1, self.num_nodes_in_layers[1]))
self.weight2 = np.random.normal(0, 1, [self.num_nodes_in_layers[1],
self.num_nodes_in_layers[2]])
self.bias2 = np.zeros((1, self.num_nodes_in_layers[2]))接下來，先將input輸入到neural network中，算出z1, a1, z2, y。
z1 = np.dot(inputs_batch, self.weight1) + self.bias1
a1 = function.relu(z1)
z2 = np.dot(a1, self.weight2) + self.bias2
y = function.softmax(z2)在使用gradient descent前，必須要先用backpropagation來算出weight和bias的gradient。
# backward pass
delta_y = (y - labels_batch) / y.shape[0]
delta_hidden_layer = np.dot(delta_y, self.weight2.T)
delta_hidden_layer[a1 <= 0] = 0 # derivatives of relu# backpropagation
weight2_gradient = np.dot(a1.T, delta_y) # forward * backward
bias2_gradient = np.sum(delta_y, axis = 0, keepdims = True)
weight1_gradient = np.dot(inputs_batch.T, delta_hidden_layer)
bias1_gradient = np.sum(delta_hidden_layer, axis = 0, keepdims =
True)
有了weights和biases的gradients之後就可以透過stochastic gradient descent來不斷更新weights和biases，直到network的loss是可以接受的。
# stochastic gradient descent
self.weight1 -= self.learning_rate * weight1_gradient
self.bias1 -= self.learning_rate * bias1_gradient
self.weight2 -= self.learning_rate * weight2_gradient
self.bias2 -= self.learning_rate * bias2_gradient程式運行如下：
...=== Epoch: 5/5 Iteration:59992 Loss: 0.55 ====== Epoch: 5/5 Iteration:59993 Loss: 0.73 ====== Epoch: 5/5 Iteration:59994 Loss: 0.54 ====== Epoch: 5/5 Iteration:59995 Loss: 0.60 ====== Epoch: 5/5 Iteration:59996 Loss: 0.43 ====== Epoch: 5/5 Iteration:59997 Loss: 0.41 ====== Epoch: 5/5 Iteration:59998 Loss: 0.49 ====== Epoch: 5/5 Iteration:59999 Loss: 0.73 ====== Epoch: 5/5 Iteration:60000 Loss: 0.88 ===Testing...Test accuracy: 91.540%
經過5次epoch，每個epoch共60000次迭代，最後得到accuracy=91.54%
loss的變化如下圖，從150下降到0.6左右。
測試結果：
完整程式碼：https://github.com/gary30404/neural-network-from-scratch-python
If you like this article and consider it useful for you, please support it with 👏.
