the open
    research
    Contribute
    Back to the reading lists

    Deep learning· 2016

    SGDR: Stochastic Gradient Descent with Warm Restarts

    Ilya Loshchilov (first author) · 2016

    Read the original

    arXiv 1608.03983OpenAlex W2518108298

    On the reading lists

    Why it matters

    Popularized cosine learning-rate annealing with warm restarts for neural network optimization.

    Condensed from the expert lists' notes.

    The record

    First released
    2016 (from arXiv 1608.03983)
    Citations
    11,568 citations, as counted by Papers With Code (CC BY-SA 4.0)
    Named by
    both expert lists
    Score
    0.369 · computed rank 430 · consensus 0.32, must-read 0, citations percentile 0.779 (2010s · deep-learning), awards 0, existing lists 0. Method

    Data sources

    Catalogue data from OpenAlex (CC0), Crossref (CC0 metadata; no abstracts are shown), arXiv, and Papers With Code (CC BY-SA 4.0). Each count names its source. See the method.