AIpedia

The 2017 paper that introduced the Transformer architecture, replacing recurrence with self-attention

A framework that bridges reinforcement learning and maximum likelihood estimation for sampling-based tasks with binary feedback

Adaptive learning rates with momentum for deep learning

The algorithm that enables neural networks to learn by computing gradients efficiently

Normalizing layer inputs to accelerate deep network training

Pre-training deep bidirectional representations for NLP

Eliciting step-by-step reasoning in language models for complex problem solving

Learning visual concepts from natural language supervision

Generative models that learn to denoise, enabling high-quality image and video synthesis

Combining Q-learning with deep neural networks for Atari-level game playing

Randomly dropping units during training to prevent overfitting

Two neural networks compete to generate realistic data

Autoregressive language models that learn to predict the next token

How large language models learn from examples in the prompt without weight updates

High-resolution image generation by diffusing in learned latent spaces

Normalizing across features for sequence models and Transformers

Linear-time sequence modeling as an efficient alternative to Transformers

Directly optimizing policies through gradient ascent on expected returns

A stable, sample-efficient policy gradient algorithm for reinforcement learning

A paradigm where LLMs treat context as an environment and recursively call themselves on sub-problems

Learning optimal behavior through interaction with an environment

Aligning language models with human preferences through reward modeling

Encoder-decoder architecture for mapping sequences to sequences

Applying Transformers directly to image patches for visual recognition

Learning dense vector representations of words from text

The deep CNN that won ImageNet 2012 and sparked the deep learning revolution

Line-by-line PyTorch implementation of the Transformer architecture

The paper that introduced the attention mechanism for sequence-to-sequence models

The Coffee Automaton paper formalizing how complexity peaks then declines

Why complexity rises then falls while entropy only increases

Stanford's foundational course on deep learning for computer vision

Scaling up end-to-end speech recognition with RNNs and CTC

Expanding receptive fields exponentially without losing resolution or adding parameters

Training giant neural networks by pipelining micro-batches across devices

The mathematical foundation for measuring information content and randomness

Shane Legg's PhD thesis formalizing universal intelligence and the AIXI agent

Grünwald's comprehensive guide to MDL for model selection and learning

Hinton's MDL approach to neural network regularization through noisy weights

Neural networks augmented with external memory and attention-based read/write heads

A unified framework for graph neural networks applied to molecular property prediction

Neural architecture that outputs pointers to input positions, enabling variable-size outputs

How input and output ordering affects seq2seq learning on set-structured data

Relation Networks for learning to reason about object relationships

RNNs with relational memory that enables reasoning across time

Pre-activation ResNet design that enables training of 1000+ layer networks

Andrej Karpathy's influential blog post demonstrating RNN capabilities through character-level generation

How to apply dropout to LSTMs without disrupting memory dynamics

Empirical laws governing how language model performance scales with compute, data, and parameters

Deep residual learning with skip connections that enabled training of 152+ layer networks

The Transformer architecture that replaced recurrence with self-attention

Christopher Olah's visual guide to Long Short-Term Memory networks

Probabilistic generative model with structured latent space

Connecting VAEs to lossy compression and the bits-back coding argument

Sequential tree ensembles optimized via gradient descent

Differentiable decision trees and oblivious ensembles for tabular learning

The initial phase of training foundation models on vast amounts of data

Finding stable matchings via the Gale–Shapley algorithm

Open-domain platform for web-based reinforcement learning agents