Attention is all you need

Recurrent models typically factor computation along the symbol positions of the input and output sequences. RNNs process a sequence one position/token at a time, rather than processing all positions