Paper2017
Attention Is All You Need
Vaswani et al.
The 2017 paper that introduced the Transformer architecture
Transformers
8 entries / Filter in the index →
Paper2020
Vision Transformer (ViT)
Dosovitskiy et al.
Applying Transformers directly to image patches for visual recognition
Paper2018
BERT: Bidirectional Transformers
Devlin et al.
Pre-training deep bidirectional representations for NLP
Paper2018
GPT: Generative Pre-Training
Radford et al.
Autoregressive language models that learn to predict the next token
Paper2020
Scaling Laws for Neural Language Models
Kaplan et al.
Why bigger models, more data, and more compute lead to predictable gains
Blog2018
The Annotated Transformer
Sasha Rush
Line-by-line PyTorch implementation of the Transformer architecture
