Vision Transformers

Yann LeCun's tweet is spot on. My final year project focuses on vision transformers, and I can attest that they excel in vision tasks, possibly even surpassing convolutional neural networks (convnets). However, transformers demand significant resourc...