Peer-reviewedReviewed
Sequence to Sequence Learning with Neural Networks
Introduced a general end-to-end LSTM encoder-decoder that maps an input sequence to a fixed-length vector and decodes it into an output sequence, removing the need for fixed-alignment, feature-engineered pipelines in sequence transduction.
Executive summary
The paper used one multilayer LSTM to read an entire source sentence into a single vector and a second LSTM to generate the target sentence from that vector. A key practical trick was reversing the order of source words, which shortened the average distance between corresponding input and output tokens and made optimization easier, pushing WMT'14 English-to-French BLEU to competitive levels. It showed that a purely neural, general-purpose sequence model could rival phrase-based statistical MT and became the template for later encoder-decoder systems.
Antecedents
- applies to Long Short-Term Memory — Seq2seq built from LSTMsStrongly supported
- combines Gated Recurrent Unit / RNN Encoder-Decoder — GRU encoder-decoder sibling of seq2seqStrongly supported
What it enabled — descendants
- extends Attention Is All You Need — Encoder-decoder transduction framing carried over from seq2seqDirect evidence
- extends Neural Machine Translation by Jointly Learning to Align and Translate — Attention removes seq2seq bottleneckDirect evidence