Transformers

Transformers 2017: Attention Is All You Need

Vaswani et al. (Google) replaced recurrence with self-attention—the base architecture of BERT, GPT, and nearly all modern NLP.