12 · CONCLUSION

Conclusion & Legacy

The Transformer: the first sequence transduction model based entirely on attention — multi-headed self-attention replaces recurrent layers in encoder–decoder architectures.
Trains significantly faster than recurrent or convolutional architectures thanks to full parallelization.
New state of the art on WMT 2014 EN→DE (28.4 BLEU) and EN→FR (41.8 BLEU), at a fraction of the previous training cost.
The architecture became the foundation of modern large language models (BERT, GPT, and their successors) — attention really was all you needed.

“We are excited about the future of attention-based models and plan to apply them to other tasks.” — Vaswani et al., §7 Conclusion (paper p. 10)

Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin. “Attention Is All You Need.” NeurIPS 2017 (arXiv:1706.03762). All figures/tables shown are cropped from the original paper PDF.