AIpedia

The deep CNN that won ImageNet 2012 and sparked the deep learning revolution

Line-by-line PyTorch implementation of the Transformer architecture

The paper that introduced the attention mechanism for sequence-to-sequence models

The Coffee Automaton paper formalizing how complexity peaks then declines

Why complexity rises then falls while entropy only increases

Stanford's foundational course on deep learning for computer vision

Scaling up end-to-end speech recognition with RNNs and CTC

Expanding receptive fields exponentially without losing resolution or adding parameters

Training giant neural networks by pipelining micro-batches across devices

The mathematical foundation for measuring information content and randomness

Shane Legg's PhD thesis formalizing universal intelligence and the AIXI agent

Grünwald's comprehensive guide to MDL for model selection and learning

Hinton's MDL approach to neural network regularization through noisy weights

A unified framework for graph neural networks applied to molecular property prediction

Neural networks augmented with external memory and attention-based read/write heads

How input and output ordering affects seq2seq learning on set-structured data

Neural architecture that outputs pointers to input positions, enabling variable-size outputs

Relation Networks for learning to reason about object relationships

RNNs with relational memory that enables reasoning across time

Pre-activation ResNet design that enables training of 1000+ layer networks

Deep residual learning with skip connections that enabled training of 152+ layer networks

Andrej Karpathy's influential blog post demonstrating RNN capabilities through character-level generation

How to apply dropout to LSTMs without disrupting memory dynamics

Empirical laws governing how language model performance scales with compute, data, and parameters

The Transformer architecture that replaced recurrence with self-attention

Christopher Olah's visual guide to Long Short-Term Memory networks

Probabilistic generative model with structured latent space

Connecting VAEs to lossy compression and the bits-back coding argument

Sequential tree ensembles optimized via gradient descent

Differentiable decision trees and oblivious ensembles for tabular learning

The initial phase of training foundation models on vast amounts of data

Finding stable matchings via the Gale–Shapley algorithm

Open-domain platform for web-based reinforcement learning agents