Aug 31, 2018 · 1 min read
Hey thanks for reading the article.
For the Relu’s derivative: x =1 if x>0 and it is 1 in the graph too. The reason why you think it is reaching infinity because the y axis is also ends at 1. So the Black outline of the graph is overlapping with the Relu’s derivative line which took a right turn when reaching 1 on the y axis. Look closely.

Here is another one form google for reference.
Thanks for the asking Ben Bradfield
