AIpedia

Start here

New to the field? Follow this path from fundamentals to modern models.

1. 1 Backpropagation The algorithm that enables neural networks to learn by computing gradients efficiently
2. 2 Understanding LSTM Networks Christopher Olah's visual guide to Long Short-Term Memory networks
3. 3 Sequence to Sequence Learning Encoder-decoder architecture for mapping sequences to sequences
4. 4 Neural Machine Translation by Jointly Learning to Align and Translate The paper that introduced the attention mechanism for sequence-to-sequence models
5. 5 Transformer Self-attention models that process sequences in parallel
6. 6 GPT: Generative Pre-Training Autoregressive language models that learn to predict the next token
7. 7 Reinforcement Learning Learning by trial and error through rewards
8. 8 Policy Gradient Methods Directly optimizing policies through gradient ascent on expected returns

Recently added

• Attention Is All You Need Feb 26, 2026 The 2017 paper that introduced the Transformer architecture
• Maximum Likelihood Reinforcement Learning (MaxRL) Feb 13, 2026 A recent idea for training models on pass-fail tasks when sampling matters
• Adam Optimizer Feb 1, 2026 Adaptive learning rates with momentum for deep learning
• Backpropagation Feb 1, 2026 The algorithm that enables neural networks to learn by computing gradients efficiently
• Batch Normalization Feb 1, 2026 Normalizing layer inputs to accelerate deep network training
• BERT: Bidirectional Transformers Feb 1, 2026 Pre-training deep bidirectional representations for NLP
• CLIP: Contrastive Language-Image Pre-training Feb 1, 2026 Learning visual concepts from natural language supervision
• Chain-of-Thought Prompting Feb 1, 2026 Eliciting step-by-step reasoning in language models for complex problem solving
• Diffusion Models Feb 1, 2026 Generative models that learn to denoise, enabling high-quality image and video synthesis
• Deep Q-Networks (DQN) Feb 1, 2026 Combining Q-learning with deep neural networks for Atari-level game playing