Following One Sentence Through a Transformer: A Tensor-Shape Walkthrough

Diagrams of the Transformer show boxes and arrows. What helped me most was following a single sentence through the model and writing down the shape of the data at every step. Here is that trace, using