Architecture researcherScientific originator
Noam Shazeer
Co-invented the Transformer and later SwiGLU, multi-query attention, and sparse mixture-of-experts routing.
Works in the archive (7) · lead author on 4
- Attention Is All You Need · 2017
- LeadOutrageously Large Neural Networks: Sparsely-Gated MoE · 2017
- LeadMesh-TensorFlow / GSPMD · 2018
- LeadFast Transformer Decoding: One Write-Head is All You Need (MQA) · 2019
- LeadGLU Variants Improve Transformer (SwiGLU) · 2020
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (T5) · 2020
- Switch Transformer · 2022
“Lead” marks the first-listed author — a byline fact, not a measure of influence; co-authorship is shown as recorded.
Co-authors in the archive
View co-author network (25 people)
Loading graph…