Topics
58 articles across 84 tags. Press ⌘K to search.
Main tags
All articles
A
- A Simple Neural Network Module for Relational Reasoning Relation Networks for learning to reason about object relationships
- A Tutorial Introduction to the Minimum Description Length Principle Grünwald's comprehensive guide to MDL for model selection and learning
- Adam Optimizer Adaptive learning rates with momentum for deep learning
- AlexNet The deep CNN that won ImageNet 2012 and sparked the deep learning revolution
- Attention Is All You Need The 2017 paper that introduced the Transformer architecture
B
C
- Chain-of-Thought Prompting Eliciting step-by-step reasoning in language models for complex problem solving
- CLIP: Contrastive Language-Image Pre-training Learning visual concepts from natural language supervision
- CS231n: CNNs for Visual Recognition Stanford's foundational course on deep learning for computer vision
D
- Deep Q-Networks (DQN) Combining Q-learning with deep neural networks for Atari-level game playing
- Deep Speech 2: End-to-End Speech Recognition Scaling up end-to-end speech recognition with RNNs and CTC
- Diffusion Models Generative models that learn to denoise, enabling high-quality image and video synthesis
- Dropout: Regularization for Neural Networks Randomly dropping units during training to prevent overfitting
G
- Generative Adversarial Networks Two neural networks compete to generate realistic data
- GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism Training giant neural networks by pipelining micro-batches across devices
- GPT: Generative Pre-Training Autoregressive language models that learn to predict the next token
- Gradient Boosted Decision Trees Sequential tree ensembles optimized via gradient descent
I
K
L
M
- Machine Super Intelligence Shane Legg's PhD thesis formalizing universal intelligence and the AIXI agent
- Mamba: State Space Models A sequence model that keeps a running state instead of attending to every token pair
- Maximum Likelihood Reinforcement Learning (MaxRL) A recent idea for training models on pass-fail tasks when sampling matters
- Multi-Scale Context Aggregation by Dilated Convolutions Expanding receptive fields exponentially without losing resolution or adding parameters
N
- Neural Machine Translation by Jointly Learning to Align and Translate The paper that introduced the attention mechanism for sequence-to-sequence models
- Neural Message Passing for Quantum Chemistry A unified framework for graph neural networks applied to molecular property prediction
- Neural Turing Machines Neural networks augmented with external memory and attention-based read/write heads
- NODE (Neural Oblivious Decision Ensembles) Differentiable decision trees and oblivious ensembles for tabular learning
P
- Pointer Networks Neural architecture that outputs pointers to input positions, enabling variable-size outputs
- Policy Gradient Methods Directly optimizing policies through gradient ascent on expected returns
- Pre-training The stage where a model learns broad patterns from a very large dataset
- Proximal Policy Optimization (PPO) A stable, sample-efficient policy gradient algorithm for reinforcement learning
R
- Recurrent Neural Network Regularization How to apply dropout to LSTMs without disrupting memory dynamics
- Recursive Language Models A paradigm where LLMs treat context as an environment and recursively call themselves on sub-problems
- Reinforcement Learning Learning by trial and error through rewards
- Relational Recurrent Neural Networks RNNs with relational memory that enables reasoning across time
- ResNet Deep residual learning with skip connections that enabled training of 152+ layer networks
- RLHF: Reinforcement Learning from Human Feedback Teaching language models to prefer responses that people rank higher
S
- Scaling Laws for Neural Language Models Why bigger models, more data, and more compute lead to predictable gains
- Sequence to Sequence Learning Encoder-decoder architecture for mapping sequences to sequences
- Stable Marriage Problem Finding a stable matching with the Gale-Shapley deferred acceptance algorithm
T
- The Annotated Transformer Line-by-line PyTorch implementation of the Transformer architecture
- The First Law of Complexodynamics Why complexity rises then falls while entropy only increases
- The Unreasonable Effectiveness of Recurrent Neural Networks Andrej Karpathy's influential blog post demonstrating RNN capabilities through character-level generation
- Transformer Self-attention models that process sequences in parallel