Aug 9, 2017 · 1 min read
Hi guys great tutorial and the mathematics was very clear. However on the code side I am confused as to what you define as the inputs and outputs of the layer (Is the input the scalar value per neuron of the combined weights or are they each weight multiplied by previous neurons? Is the output just the scalar value per neuron after the activation?). Specifically in the API design you mentioned:
#out_grad is the derivative of the cost function w.r.t. the
# inputs to all of the neurons for the following layer.but then it seems to me in
output_wrt_inputs = self.W
output_wrt_inputs[:, self.out_act < 0] = 0cost_wrt_inputs = cost_wrt_output * output_wrt_inputsreturn cost_wrt_inputs
there is either a contradictory way of defining out_grad or the meaning of ‘inputs’ is different. Would you be able to clarify further what is going on in this code?
