The Kaiming initialization and Batch Normalization in Neural Networks

Jan 7, 2025 · 17 min read · In my last blog post, we discussed and solved through code, how to initialize the hidden layer in neural network properly, to get rid of dead neurons. We came up with some magic numbers, randomly, as well as, by looking at the mathematics of tanh act...