the open
    research
    Contribute
    Back to the reading lists

    Language, speech & LLMs· 2019

    RoBERTa: A Robustly Optimized BERT Pretraining Approach

    Yinhan Liu (first author) · 2019

    Read the original

    arXiv 1907.11692OpenAlex W2965373594

    On the reading lists

    Why it matters

    Showed stronger training procedures could dramatically improve BERT without fundamental architectural changes.

    Condensed from the expert lists' notes.

    The record

    First released
    2019 (from arXiv 1907.11692)
    Citations
    31,842 citations, as counted by Papers With Code (CC BY-SA 4.0)
    Named by
    both expert lists
    Score
    0.501 · computed rank 234 · consensus 0.53, must-read 0, citations percentile 0.978 (2010s · language), awards 0, existing lists 0. Method

    Data sources

    Catalogue data from OpenAlex (CC0), Crossref (CC0 metadata; no abstracts are shown), arXiv, and Papers With Code (CC BY-SA 4.0). Each count names its source. See the method.