Leaky ReLU: improving traditional ReLU
The Leaky ReLU is a type of activation function which comes across many machine learning blogs every now and then....
Tag: activation functions
He/Xavier initialization & activation functions: choose wisely
Deep learning models require to be initialized. Their layers have activation functions to make neuron outputs nonlinear. But how to...
Implementing ReLU, Sigmoid and Tanh in Keras
In a recent blog post, we looked at widely used activation functions in today’s neural networks. More specifically, we checked...
ReLU, Sigmoid and Tanh: today’s most used activation functions
Today’s deep neural networks can handle highly complex data sets. For example, object detectors have grown capable of predicting the...
Why you shouldn’t use a linear activation function
In today’s deep learning community, three activation functions are commonly used: the sigmoid function, the tanh function and the Rectified...
What is deep learning exactly?
Recently, I’ve picked up deep learning both in my professional and spare-time activities. This means that I spent a lot...
