Peer-reviewed2017
Attention Is All You Need
Introduced the Transformer, a sequence model built entirely on self-attention that removed the sequential recurrence of RNNs and enabled full parallelization of training across positions.
An evidence-led archive
The papers, systems, and ideas that built modern AI.
A map of the research behind Transformers, scaling, alignment, reasoning, multimodality, retrieval, and agents — organised by how each work depends on the others, not by fame or citation count.
- Records
- 211
- Reviewed
- 211
- Dependency links
- 198
Every record links a primary source and has been editorially reviewed. Citation counts are estimates, shown with their retrieval date and never used to rank. See the methodology.
Featured canonical works
Browse all 211 →Peer-reviewed2020
Language Models are Few-Shot Learners (GPT-3)
Showed that scaling an autoregressive language model to 175B parameters yields in-context few-shot learning, where tasks are specified through examples in the prompt rather than by fine-tuning weights.
Peer-reviewed2022
Training Compute-Optimal Large Language Models (Chinchilla)
Corrected the earlier scaling prescription by showing parameters and training tokens should be scaled in roughly equal proportion, revealing that then-current large models were badly undertrained for their compute budgets.
Peer-reviewed2022
Chain-of-Thought Prompting Elicits Reasoning
Showed that prompting a large model to emit intermediate reasoning steps before its answer unlocks multi-step reasoning that direct-answer prompting fails at, without any fine-tuning.
Browse by research domain
Origins & ComputabilityThe people and ideas that defined the space before deep learning.Neural FoundationsBackpropagation, embeddings, recurrence, and attention before 2017.Transformer ArchitectureThe self-attention block and the components that make it work.Decoder-Only Language ModelsThe autoregressive lineage that scaled into general few-shot learners.EncodersMasked language modelling for understanding tasks.Encoder–DecodersText-to-text denoising and sequence transduction.Scaling Laws & ComputeHow loss falls with parameters, data, and compute.Data, Corpora & TokenizationThe corpora and tokenizers that models actually learn from.Distributed TrainingParallelism and memory systems that make large training feasible.Mixture-of-ExpertsConditional computation for scale without proportional cost.Alignment & Preference LearningTurning next-token predictors into steerable assistants.Reasoning & Test-Time ComputeEliciting and scaling multi-step reasoning at inference.Agents & Tool UseModels that plan, call tools, and act over environments.MultimodalityVision, audio, and video joined to language.Retrieval & MemoryGrounding generation in external, updatable knowledge.Code ModelsProgram synthesis, fill-in-the-middle, and repo-scale context.Inference & ServingMaking trained models fast and cheap to run.Long Context & Efficient SequencesExtending and exploiting the context window.Evaluation & BenchmarksHow capability is measured — and where measures break.Interpretability & SafetyReverse-engineering what trained models actually compute.Continual Learning & MemoryLearning after deployment without forgetting — consolidation and test-time memory.Open-Weight Model FamiliesDownloadable model families and what they inherit.
The dependency network
Explore the interactive graph →Modern AI was not created by one paper or one lab. It emerged from a network of architectural breakthroughs, training methods, data practices, infrastructure, alignment techniques, and open implementations. This is its spine.
Recently verified
Foundational paper1986
Backpropagation (Learning representations by back-propagating errors)
Introduced the backpropagation algorithm for training multi-layer neural networks by propagating output error backward through the network to compute weight gradients, removing the barrier that networks with hidden layers had no practical learning rule.
Peer-reviewed2003
A Neural Probabilistic Language Model
Proposed learning a distributed vector representation for each word jointly with a neural network that predicts the next word, addressing the curse of dimensionality that made count-based n-gram language models generalize poorly to unseen word combinations.
Preprint2013
Efficient Estimation of Word Representations (Word2Vec)
Introduced the continuous bag-of-words and skip-gram models (Word2Vec), simple shallow architectures that learn word vectors from raw text far more cheaply than prior neural language models, removing the compute cost that had limited embedding training to small corpora.
Peer-reviewed2014
GloVe: Global Vectors for Word Representation
GloVe introduced a word-embedding method that learns vectors by factorizing a corpus-wide word co-occurrence matrix, solving the problem of combining global corpus statistics with the useful linear structure of local-context methods.
Peer-reviewed2014
Sequence to Sequence Learning with Neural Networks
Introduced a general end-to-end LSTM encoder-decoder that maps an input sequence to a fixed-length vector and decodes it into an output sequence, removing the need for fixed-alignment, feature-engineered pipelines in sequence transduction.