The Need for Speed: Mastering Momentum, RMSprop, and the Adam Optimizer

In our previous post, we prevented our Deep Neural Networks from crashing by using mathematical tricks like He Initialization and Inverted Dropout. Our model is now stable, but there is still a massiv