the open
    research
    Contribute
    Back to the reading lists

    Deep learning· 2020

    GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

    Dmitry Lepikhin (first author) · 2020

    Read the original

    arXiv 2006.16668OpenAlex W3040573126

    On the reading lists

    Why it matters

    Combined compiler-assisted sharding with conditional computation to train enormous distributed MoE models.

    Condensed from the expert lists' notes.

    The record

    First released
    2020 (from arXiv 2006.16668)
    Citations
    2,467 citations, as counted by Papers With Code (CC BY-SA 4.0)
    Named by
    both expert lists
    Score
    0.314 · computed rank 572 · consensus 0.5, must-read 0, citations percentile 0.469 (2020s · deep-learning), awards 0, existing lists 0. Method

    Data sources

    Catalogue data from OpenAlex (CC0), Crossref (CC0 metadata; no abstracts are shown), arXiv, and Papers With Code (CC BY-SA 4.0). Each count names its source. See the method.