Batch Norm vs Layer Norm
Summary:
Batch Norm -> normalizing each feature across batch.
Layer Norm -> normalizing the values of each individual example independently.
Deep Dive:
Suppose we have 4 students (examples / rows)
ug-beast.hashnode.dev2 min read