the open
    research
    Contribute
    Back to the reading lists

    Computer vision & multimodal· 2021

    Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

    Junnan Li (first author) · 2021

    Read the original

    arXiv 2107.07651OpenAlex W3184735396

    On the reading lists

    Why it matters

    Combined contrastive alignment, cross-modal fusion, and momentum distillation for robust vision-language pretraining.

    Condensed from the expert lists' notes.

    The record

    First released
    2021 (from arXiv 2107.07651)
    Citations
    3,002 citations, as counted by Papers With Code (CC BY-SA 4.0)
    Named by
    both expert lists
    Score
    0.361 · computed rank 451 · consensus 0.67, must-read 0, citations percentile 0.457 (2020s · vision), awards 0, existing lists 0. Method

    Data sources

    Catalogue data from OpenAlex (CC0), Crossref (CC0 metadata; no abstracts are shown), arXiv, and Papers With Code (CC BY-SA 4.0). Each count names its source. See the method.