Paper2017
Attention Is All You Need
Vaswani et al.
The 2017 paper that introduced the Transformer architecture
NLP
15 entries / Filter in the index →
Concept2022
Chain-of-Thought Prompting
Wei et al.
Eliciting step-by-step reasoning in language models for complex problem solving
Paper2021
CLIP: Contrastive Language-Image Pre-training
Radford et al.
Learning visual concepts from natural language supervision
Concept2020
In-Context Learning
Brown et al.
How large language models learn from examples in the prompt without weight updates
Paper2018
BERT: Bidirectional Transformers
Devlin et al.
Pre-training deep bidirectional representations for NLP
Paper2018
GPT: Generative Pre-Training
Radford et al.
Autoregressive language models that learn to predict the next token
Concept2017
RLHF: Reinforcement Learning from Human Feedback
Christiano et al.
Teaching language models to prefer responses that people rank higher
Paper2014
Sequence to Sequence Learning
Sutskever, Vinyals & Le
Encoder-decoder architecture for mapping sequences to sequences
Concept2013
Word2Vec: Word Embeddings
Mikolov et al.
Learning dense vector representations of words from text
Paper2020
Scaling Laws for Neural Language Models
Kaplan et al.
Why bigger models, more data, and more compute lead to predictable gains
Blog2018
The Annotated Transformer
Sasha Rush
Line-by-line PyTorch implementation of the Transformer architecture
Blog2015
The Unreasonable Effectiveness of Recurrent Neural Networks
Andrej Karpathy
Andrej Karpathy's influential blog post demonstrating RNN capabilities through character-level generation
Blog2015
Understanding LSTM Networks
Chris Olah
Christopher Olah's visual guide to Long Short-Term Memory networks
Paper2014
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, Cho & Bengio
The paper that introduced the attention mechanism for sequence-to-sequence models