Differentiating the Sigmoid Function

Every data scientist is learning, and in some cases specializing. In the last year I’ve developed some expertise in distributed computing with Apache Spark, and now I’m pushing into Deep Learning through Andrew Ng’s Deep Learning Course.

Today we were drawing out some simple computation graphs while defining a cost function for logistic regression.

Andrew Ng | deeplearning.io @ Coursera

Where our loss function is defined as:

And the sigmoid function is of course:

So how do we show that da/dz = a*(1-a), where a is the sigmoid function defined above?

Note: Medium has no support for latex or math expressions at the time of writing, so I’m using my hand notes below. Sorry for potato quality!

This is quite simple to do with a few tricks. Here are the steps I’ll use:

• Re-write sigmoid as a composite function, expressing the denominator as f(x).
• I’ll then use the chain rule to differentiate, expressing the derivative in terms of f(x).
• Lastly, I’ll differentiate f(x), and re-write the result as in terms of f(x) itself. When we substitute this into the equation above, we’ll get the the result we’re after.

My last step was to express the sigmoid function in terms of a/z, like Andrew Ng had done. Fun Saturday stuff :)