Josh Weston
Aug 31, 2018 · 1 min read

I am curious about how you implement your biases (c and b). I have typically used a single bias per neuron on the hidden/output layers:

bias_1 = tf.Variable(tf.zeros(2)) # 2 neurons = 2 biases for the hidden layer

bias_2 = tf.Variable(tf.zeros(1)) # 1 neuron = 1 bias for the output layer

In your example, it appears as though you are implementing a bias for each output element of the hidden/output layers:

c = tf.Variable(tf.zeros([4,2]), name = “c”)

b = tf.Variable(tf.zeros([4,1]), name = “b”)

Can you explain what the advantages/disadvantages are of either approach? I would think your approach allows for more degrees of freedom for the network to learn with. Perhaps my approach is flawed?

    Josh Weston

    Written by