A Concise Introduction to the ReLU Function

Introduction The rectified linear unit function, ReLU, is one of the three most commonly used activation functions used in deep learning, including the sigmoid (sig) and hyperbolic tangent (tan) functions. It is a type of ramp function that is somewh...