Paper2017
Attention Is All You Need
Vaswani et al.
The 2017 paper that introduced the Transformer architecture
Architecture
8 entries / Filter in the index →
Paper2023
Mamba: State Space Models
Gu & Dao
A sequence model that keeps a running state instead of attending to every token pair
Paper2018
BERT: Bidirectional Transformers
Devlin et al.
Pre-training deep bidirectional representations for NLP
Paper2018
GPT: Generative Pre-Training
Radford et al.
Autoregressive language models that learn to predict the next token
Concept2015
Batch Normalization
Ioffe & Szegedy
Normalizing layer inputs to accelerate deep network training
Concept2014
Generative Adversarial Networks
Goodfellow et al.
Two neural networks compete to generate realistic data
Paper2014
Sequence to Sequence Learning
Sutskever, Vinyals & Le
Encoder-decoder architecture for mapping sequences to sequences
