the open
    research
    Contribute
    Back to the catalogue

    Language models · 2020

    Language Models are Few-Shot Learners

    Brown et al.

    Read the original
    Three examples, then your turn.

    The idea, in brief

    Scale language models and evaluate task performance using examples in the prompt.

    An exploration of what a large language model can learn from a handful of examples in its context.

    Read with a question in mind.

    What evidence supports the main claim? Which assumptions would change the result? What would it take to reproduce it?

    Go to the source

    This introduction is an editorial summary. The original paper contains the methods, experiments, and limitations you need for a closer look.

    arXiv: 2005.14165

    On the reading lists

    Why it matters

    Established large-scale language models as general-purpose few-shot learners without parameter updates.

    Condensed from the expert lists' notes.

    Inside the paper

    The problem
    Adapting pretrained language models to a new task often required a separate labeled dataset and a fine-tuning run. The authors ask how far a model can go when a task is specified in text at inference time.
    The idea
    Train a family of autoregressive language models and evaluate them using instructions and examples placed in the input. The model continues the prompt without changing its parameters for the task.
    How it works
    • The models learn to predict the next token from text drawn from a mixture of sources, including filtered web data.
    • At evaluation time a zero-shot prompt describes the task, while one-shot and few-shot prompts also supply examples.
    • The model completes a new example from the same prompt; the authors compare several model sizes and prompt conditions.
    • They check benchmark overlap with training data and report limitations where contamination could affect the interpretation of results.
    What it showed
    The largest model performs well on several question answering, translation and completion tests, although it remains weak on some tasks. Few-shot gains often grow with model size, but performance and contamination vary by benchmark.
    Read it for
    Figure 2.1 makes the evaluation settings concrete; read Sections 3.2–3.3 for examples and Section 4 for contamination. The full list of prompts in the appendix is useful later.

    Written in our own words from the paper itself. A draft the editors have not yet reviewed.

    Reading it

    Difficulty: Intermediate

    Read this first: the Transformer (Attention Is All You Need); language modelling basics.

    The record

    First released
    2020 (from arXiv 2005.14165)
    Citations
    64,736 citations, as counted by Papers With Code (CC BY-SA 4.0)
    Named by
    both expert lists
    Score
    0.744 · computed rank 36 · consensus 0.98, must-read 1, citations percentile 1 (2020s · language), awards 0, existing lists 0. Method

    Editor's notes

    • Readability check: pass: Defines in-context learning; read the first sections, skim the evaluations. (AI draft (Claude), not yet reviewed, 2026-10-29)

    This paper's reading-list entry has its rank, release date and sources in full.

    Data sources

    Catalogue data from OpenAlex (CC0), Crossref (CC0 metadata; no abstracts are shown), arXiv, and Papers With Code (CC BY-SA 4.0). Each count names its source. See the method.