Derivatives Gradients and Backpropagation Interconnection

In this post, we will try to answer in simple terms the relation between derivatives, gradients and backpropagation in neural network circuits. What are derivatives ? Derivatives denotes the magnitude and direction of slopes, which indicates the rate...