Brown et al. showed a large enough LLM performs tasks with few-shot prompts without task-specific fine-tuning.
In May 2020, OpenAI published the GPT-3 paper (175B). The core finding: at sufficient scale, the model executes many tasks via prompt demonstrations.
OpenAI’s API industrialized access. Startups built products on an endpoint. Limits appeared: cost, hallucinations, bias.
GPT-3 shifted NLP’s center of gravity from ubiquitous fine-tuning toward prompting and later instruction and chat.
OpenAI presented GPT-2 and initially limited release of the largest model over disinformation risks, foreshadowing open vs closed dilemmas.
OpenAI launched ChatGPT and reached 100 million users in months; conversational AI became a global habit.
OpenAI launched GPT-4 with notable gains on professional exams, robustness, and later visual ability.