Traversing the intricate expanses of multidimensional manifolds, transmuting probabilistic turbulence into infinite sagacity.
Nothing here yet.
Jun 25, 2025 · 4 min read · In the previous post, we looked at Softmax and NLL loss, both critical for output interpretation and learning in Transformers. Now let’s dive into what happens within the network: activation functions. Specifically, GELU. What is GeLU? GeLU, or, Gau...
Join discussion
May 30, 2025 · 6 min read · In the heart of modern neural networks capable of understanding the sentence semantics and generating thousands of words per second, Transformers, as we call them, lie 2 core mathematical operations that often go unnoticed, namely the Softmax and Neg...
Join discussion