Optimizations in deep learning

Gradient descent (GD) is a simple optimization method used in machine learning. When gradient steps are taken with respect to all m examples at each step, this process is referred to as Batch Gradient Descent. Typically, the most common approach for ...