RRishi·Jul 22Towards the Modern Transformer ArchitectureA deep dive into how GPT-1's architecture evolved: pre-norm, RMSNorm, SwiGLU, RoPE, and GQA — with the math behind every change.36 min read·rishi2220.hashnode.dev11K
JJessen·Apr 1, 2025Positional Encoding from Sinusoidal to RoPEUnderstanding Positional Encoding: Its Purpose and Implementation Explained16 min read·descendingnotebooks.com