Edit: A much better comparison of Swish with ReLUs is available on the second part of this blog post.
By Lukasz Burzawa, Abhishek Chaurasia and Eugenio Culurciello
We’ve reached a tipping point where it is now high time that we start the…
I originally wrote this piece as an internal R&D direction document for my colleagues at doc.ai. We’ve decided to share it publicly because we feel that it’s important that as many people as possible are thinking about these issues.