Devlin et al. released BERT, popularizing language-model fine-tuning via masked LM and next sentence prediction.
In October 2018, Google published BERT: Bidirectional Encoder Representations from Transformers. Masked pretraining enabled rich bidirectional contexts.
Fine-tuning BERT smashed records on GLUE and SQuAD. Industry adopted “download checkpoint + adapt” as the standard flow.
BERT fixed the encoder paradigm; GPT fixed the generative decoder. Tension between them shaped products through the ChatGPT era.
Vaswani et al. (Google) replaced recurrence with self-attention—the base architecture of BERT, GPT, and nearly all modern NLP.
Google merged Brain/DeepMind efforts into Gemini, a multimodal family natively trained for text, code, and image.
Tomas Mikolov and Google colleagues released Word2Vec, popularizing dense embeddings and king–man+woman≈queen analogies.