RRishiJul 22 · 36 min readTowards the Modern Transformer ArchitectureHi y'all, Long time no see! Last post was last year. Everyone has likely read Attention Is All You Need, and understood everything around self-attention, and mapped out how encoders and decoders talk 11Krishi2220.hashnode.dev
JJessenApr 1, 2025 · 16 min readPositional Encoding from Sinusoidal to RoPETransformers process the tokens of a text input in parallel, but unlike sequential models they do not understand position and see the input as a set of tokens. However when we calculate attention for a sentence, words that are the same but in differe...descendingnotebooks.com
