Aug 31, 2018 · 1 min read
I am curious about how you implement your biases (c and b). I have typically used a single bias per neuron on the hidden/output layers:
bias_1 = tf.Variable(tf.zeros(2)) # 2 neurons = 2 biases for the hidden layer
bias_2 = tf.Variable(tf.zeros(1)) # 1 neuron = 1 bias for the output layer
In your example, it appears as though you are implementing a bias for each output element of the hidden/output layers:
c = tf.Variable(tf.zeros([4,2]), name = “c”)
b = tf.Variable(tf.zeros([4,1]), name = “b”)
Can you explain what the advantages/disadvantages are of either approach? I would think your approach allows for more degrees of freedom for the network to learn with. Perhaps my approach is flawed?