2020
GPT-3
OpenAI launches GPT-3 with 175 billion parameters: the first language model that convincingly generates human-like text, code, and translations, making the world realize what LLMs can do.
The model that changed everything
In May 2020, OpenAI published the paper Language Models are Few-Shot Learners, introducing GPT-3 with 175 billion parameters — at the time by far the largest language model ever trained. GPT-3 was remarkable not just for its size, but for its emergent capabilities: it could write coherent essays, generate working code, translate languages, answer trivia questions, and even pass the basics of reasoning tests — all without any task-specific fine-tuning, just by showing a few examples in the prompt.
Few-shot learning
The key insight of GPT-3 was few-shot learning: by providing a few examples of a task in the input prompt, GPT-3 could perform that task without updating its weights. This was fundamentally different from previous approaches, where a model had to be explicitly fine-tuned on labeled data for each new task. GPT-3 showed that a sufficiently large language model develops a kind of general problem-solving capability through language alone.
Public access and impact
OpenAI made GPT-3 available via an API in 2020, initially in private beta. Thousands of developers immediately began building applications: writing assistants, code generators, chatbots, search tools. The demos spread on social media and created widespread awareness that something qualitatively new had happened in AI. It also triggered concerns about misuse — fake news, phishing, academic fraud — and led OpenAI to adopt a careful access policy.
Sources
- Brown, T. et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020.
- Wikipedia — GPT-3