This New Optimizer Called Lion Could Replace Adam as the Go-To for Training Neural Nets

Google Brain researchers have discovered a new optimization algorithm for training deep neural networks called Lion that outperforms the popular Adam optimizer on a variety of computer vision and natural language processing tasks. In a paper publishe...