Beyond Swish: the LiSHT activation function

Deep neural networks perform linear operations to combine weight vectors with input vectors. The values that are the outputs of...

Why Swish could perform better than ReLu

Neural networks are composed of various layers of neurons. Mathematically, a neuron is nothing but the dot product between the...