AI and machine learning concepts explained with interactive visualshttps://aipedia.org/https://aipedia.org/attention-is-all-you-need/https://aipedia.org/attention-is-all-you-need/The 2017 paper that introduced the Transformer architectureThu, 26 Feb 2026 00:00:00 GMTpaperarchitectureattentiondeep-learningnlptransformerhttps://aipedia.org/maximum-likelihood-rl/https://aipedia.org/maximum-likelihood-rl/A recent idea for training models on pass-fail tasks when sampling mattersFri, 13 Feb 2026 00:00:00 GMTdeep-learningreinforcement-learningoptimizationpolicy-gradienthttps://aipedia.org/adam/https://aipedia.org/adam/Adaptive learning rates with momentum for deep learningSun, 01 Feb 2026 00:00:00 GMTdeep-learningtrainingoptimizationhttps://aipedia.org/backpropagation/https://aipedia.org/backpropagation/The algorithm that enables neural networks to learn by computing gradients efficientlySun, 01 Feb 2026 00:00:00 GMTdeep-learningfundamentalsoptimizationneural-networkshttps://aipedia.org/batch-normalization/https://aipedia.org/batch-normalization/Normalizing layer inputs to accelerate deep network trainingSun, 01 Feb 2026 00:00:00 GMTdeep-learningtrainingoptimizationarchitecturehttps://aipedia.org/bert/https://aipedia.org/bert/Pre-training deep bidirectional representations for NLPSun, 01 Feb 2026 00:00:00 GMTarchitecturetransformerdeep-learningnlppre-traininghttps://aipedia.org/clip/https://aipedia.org/clip/Learning visual concepts from natural language supervisionSun, 01 Feb 2026 00:00:00 GMTdeep-learningmultimodalcomputer-visionnlpfoundation-modelshttps://aipedia.org/chain-of-thought/https://aipedia.org/chain-of-thought/Eliciting step-by-step reasoning in language models for complex problem solvingSun, 01 Feb 2026 00:00:00 GMTdeep-learningnlplanguage-modelsreasoningpromptinghttps://aipedia.org/diffusion-models/https://aipedia.org/diffusion-models/Generative models that learn to denoise, enabling high-quality image and video synthesisSun, 01 Feb 2026 00:00:00 GMTdeep-learninggenerativecomputer-visiondiffusionfoundation-modelshttps://aipedia.org/dqn/https://aipedia.org/dqn/Combining Q-learning with deep neural networks for Atari-level game playingSun, 01 Feb 2026 00:00:00 GMTdeep-learningreinforcement-learninggamesclassichttps://aipedia.org/dropout/https://aipedia.org/dropout/Randomly dropping units during training to prevent overfittingSun, 01 Feb 2026 00:00:00 GMTdeep-learningtrainingregularizationoptimizationhttps://aipedia.org/gan/https://aipedia.org/gan/Two neural networks compete to generate realistic dataSun, 01 Feb 2026 00:00:00 GMTarchitecturedeep-learninggenerativecomputer-visionhttps://aipedia.org/gpt/https://aipedia.org/gpt/Autoregressive language models that learn to predict the next tokenSun, 01 Feb 2026 00:00:00 GMTarchitecturetransformerdeep-learningnlppre-traininggenerativehttps://aipedia.org/in-context-learning/https://aipedia.org/in-context-learning/How large language models learn from examples in the prompt without weight updatesSun, 01 Feb 2026 00:00:00 GMTdeep-learningnlplanguage-modelsemergent-abilitiesfoundation-modelshttps://aipedia.org/latent-diffusion/https://aipedia.org/latent-diffusion/High-resolution image generation by diffusing in learned latent spacesSun, 01 Feb 2026 00:00:00 GMTdeep-learninggenerativecomputer-visiondiffusionhttps://aipedia.org/layer-normalization/https://aipedia.org/layer-normalization/Normalizing each example across its featuresSun, 01 Feb 2026 00:00:00 GMTdeep-learningfundamentalstrainingtransformershttps://aipedia.org/mamba/https://aipedia.org/mamba/A sequence model that keeps a running state instead of attending to every token pairSun, 01 Feb 2026 00:00:00 GMTdeep-learningarchitecturesequence-modelingefficiencyhttps://aipedia.org/policy-gradient/https://aipedia.org/policy-gradient/Directly optimizing policies through gradient ascent on expected returnsSun, 01 Feb 2026 00:00:00 GMTdeep-learningreinforcement-learningoptimizationhttps://aipedia.org/ppo/https://aipedia.org/ppo/A stable, sample-efficient policy gradient algorithm for reinforcement learningSun, 01 Feb 2026 00:00:00 GMTdeep-learningreinforcement-learningpolicy-gradientoptimizationhttps://aipedia.org/recursive-language-models/https://aipedia.org/recursive-language-models/A paradigm where LLMs treat context as an environment and recursively call themselves on sub-problemsSun, 01 Feb 2026 00:00:00 GMTlanguage-modelsinferencelong-contextagentshttps://aipedia.org/reinforcement-learning/https://aipedia.org/reinforcement-learning/Learning by trial and error through rewardsSun, 01 Feb 2026 00:00:00 GMTdeep-learningreinforcement-learningagentsdecision-makinghttps://aipedia.org/rlhf/https://aipedia.org/rlhf/Teaching language models to prefer responses that people rank higherSun, 01 Feb 2026 00:00:00 GMTdeep-learningnlpalignmentreinforcement-learningfoundation-modelshttps://aipedia.org/seq2seq/https://aipedia.org/seq2seq/Encoder-decoder architecture for mapping sequences to sequencesSun, 01 Feb 2026 00:00:00 GMTarchitecturedeep-learningnlpencoder-decoderseq2seqrnnhttps://aipedia.org/word2vec/https://aipedia.org/word2vec/Learning dense vector representations of words from textSun, 01 Feb 2026 00:00:00 GMTnlpembeddingsdeep-learningrepresentation-learninghttps://aipedia.org/vision-transformer/https://aipedia.org/vision-transformer/Applying Transformers directly to image patches for visual recognitionSun, 01 Feb 2026 00:00:00 GMTdeep-learningcomputer-visiontransformersattentionhttps://aipedia.org/alexnet/https://aipedia.org/alexnet/The deep CNN that won ImageNet 2012 and sparked the deep learning revolutionTue, 13 Jan 2026 00:00:00 GMTdeep-learningcnncomputer-visionimagenethttps://aipedia.org/annotated-transformer/https://aipedia.org/annotated-transformer/Line-by-line PyTorch implementation of the Transformer architectureTue, 13 Jan 2026 00:00:00 GMTdeep-learningtransformerattentionnlphttps://aipedia.org/bahdanau-attention/https://aipedia.org/bahdanau-attention/The paper that introduced the attention mechanism for sequence-to-sequence modelsTue, 13 Jan 2026 00:00:00 GMTdeep-learningattentionnlptranslationhttps://aipedia.org/coffee-automaton/https://aipedia.org/coffee-automaton/The Coffee Automaton paper formalizing how complexity peaks then declinesTue, 13 Jan 2026 00:00:00 GMTcomplexityentropyautomatatheoryhttps://aipedia.org/complexodynamics/https://aipedia.org/complexodynamics/Why complexity rises then falls while entropy only increasesTue, 13 Jan 2026 00:00:00 GMTcomplexityentropytheoryphysicshttps://aipedia.org/cs231n/https://aipedia.org/cs231n/Stanford's foundational course on deep learning for computer visionTue, 13 Jan 2026 00:00:00 GMTdeep-learningcnncomputer-visioncoursehttps://aipedia.org/deep-speech-2/https://aipedia.org/deep-speech-2/Scaling up end-to-end speech recognition with RNNs and CTCTue, 13 Jan 2026 00:00:00 GMTdeep-learningspeechrnnctchttps://aipedia.org/dilated-convolutions/https://aipedia.org/dilated-convolutions/Expanding receptive fields exponentially without losing resolution or adding parametersTue, 13 Jan 2026 00:00:00 GMTdeep-learningcnncomputer-visionsegmentationhttps://aipedia.org/gpipe/https://aipedia.org/gpipe/Training giant neural networks by pipelining micro-batches across devicesTue, 13 Jan 2026 00:00:00 GMTdeep-learningdistributedtrainingparallelismhttps://aipedia.org/kolmogorov-complexity/https://aipedia.org/kolmogorov-complexity/Measuring how short the best description of an object can beTue, 13 Jan 2026 00:00:00 GMTcomplexityinformation-theorycomputationtheoryhttps://aipedia.org/machine-superintelligence/https://aipedia.org/machine-superintelligence/Shane Legg's PhD thesis formalizing universal intelligence and the AIXI agentTue, 13 Jan 2026 00:00:00 GMTagitheoryintelligenceaixihttps://aipedia.org/mdl-tutorial/https://aipedia.org/mdl-tutorial/Grünwald's comprehensive guide to MDL for model selection and learningTue, 13 Jan 2026 00:00:00 GMTmdlinformation-theorymodel-selectiontheoryhttps://aipedia.org/mdl-weights/https://aipedia.org/mdl-weights/Hinton's MDL approach to neural network regularization through noisy weightsTue, 13 Jan 2026 00:00:00 GMTdeep-learningregularizationmdlcompressionhttps://aipedia.org/neural-turing-machines/https://aipedia.org/neural-turing-machines/Neural networks augmented with external memory and attention-based read/write headsTue, 13 Jan 2026 00:00:00 GMTdeep-learningmemoryattentionreasoninghttps://aipedia.org/neural-message-passing/https://aipedia.org/neural-message-passing/A unified framework for graph neural networks applied to molecular property predictionTue, 13 Jan 2026 00:00:00 GMTdeep-learninggnnchemistrygraphshttps://aipedia.org/order-matters/https://aipedia.org/order-matters/How input and output ordering affects seq2seq learning on set-structured dataTue, 13 Jan 2026 00:00:00 GMTdeep-learningseq2seqsetsattentionhttps://aipedia.org/pointer-networks/https://aipedia.org/pointer-networks/Neural architecture that outputs pointers to input positions, enabling variable-size outputsTue, 13 Jan 2026 00:00:00 GMTdeep-learningattentionseq2seqcombinatorialhttps://aipedia.org/relational-reasoning/https://aipedia.org/relational-reasoning/Relation Networks for learning to reason about object relationshipsTue, 13 Jan 2026 00:00:00 GMTdeep-learningreasoningattentionvisual-qahttps://aipedia.org/relational-rnn/https://aipedia.org/relational-rnn/RNNs with relational memory that enables reasoning across timeTue, 13 Jan 2026 00:00:00 GMTdeep-learningrnnmemoryattentionhttps://aipedia.org/resnet-identity/https://aipedia.org/resnet-identity/Pre-activation ResNet design that enables training of 1000+ layer networksTue, 13 Jan 2026 00:00:00 GMTdeep-learningcnncomputer-visionresidualhttps://aipedia.org/resnet/https://aipedia.org/resnet/Deep residual learning with skip connections that enabled training of 152+ layer networksTue, 13 Jan 2026 00:00:00 GMTdeep-learningcnncomputer-visionresidualhttps://aipedia.org/rnn-regularization/https://aipedia.org/rnn-regularization/How to apply dropout to LSTMs without disrupting memory dynamicsTue, 13 Jan 2026 00:00:00 GMTdeep-learningrnnlstmregularizationhttps://aipedia.org/rnn-effectiveness/https://aipedia.org/rnn-effectiveness/Andrej Karpathy's influential blog post demonstrating RNN capabilities through character-level generationTue, 13 Jan 2026 00:00:00 GMTdeep-learningrnnnlpgenerativehttps://aipedia.org/scaling-laws/https://aipedia.org/scaling-laws/Why bigger models, more data, and more compute lead to predictable gainsTue, 13 Jan 2026 00:00:00 GMTdeep-learningnlpscalingtransformershttps://aipedia.org/transformer/https://aipedia.org/transformer/Self-attention models that process sequences in parallelTue, 13 Jan 2026 00:00:00 GMTarchitectureattentiondeep-learningnlptransformerhttps://aipedia.org/vae/https://aipedia.org/vae/Probabilistic generative model with structured latent spaceTue, 13 Jan 2026 00:00:00 GMTdeep-learninggenerativeautoencoderlatent-spacehttps://aipedia.org/understanding-lstms/https://aipedia.org/understanding-lstms/Christopher Olah's visual guide to Long Short-Term Memory networksTue, 13 Jan 2026 00:00:00 GMTdeep-learningrnnlstmnlphttps://aipedia.org/variational-lossy-autoencoder/https://aipedia.org/variational-lossy-autoencoder/Understanding VAEs as compression systems with a rate-distortion trade-offTue, 13 Jan 2026 00:00:00 GMTdeep-learningvaecompressiongenerativehttps://aipedia.org/gbdt/https://aipedia.org/gbdt/Sequential tree ensembles optimized via gradient descentSat, 10 Jan 2026 00:00:00 GMTmachine-learningensembleboostingtreeshttps://aipedia.org/node/https://aipedia.org/node/Differentiable decision trees and oblivious ensembles for tabular learningSat, 10 Jan 2026 00:00:00 GMTdeep-learningtreestabular-datadifferentiablehttps://aipedia.org/pre-training/https://aipedia.org/pre-training/The stage where a model learns broad patterns from a very large datasetSat, 10 Jan 2026 00:00:00 GMTdeep-learningtrainingfoundation-modelshttps://aipedia.org/stable-marriage/https://aipedia.org/stable-marriage/Finding a stable matching with the Gale-Shapley deferred acceptance algorithmSat, 10 Jan 2026 00:00:00 GMTalgorithmmatchinggame-theoryoptimizationhttps://aipedia.org/world-of-bits/https://aipedia.org/world-of-bits/Open-domain platform for web-based reinforcement learning agentsSat, 10 Jan 2026 00:00:00 GMTreinforcement-learningweb-agentsbenchmarkmultimodal