the open
    research
    Contribute
    Back to the reading lists

    Language, speech & LLMs· 2023

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    Carlos E. Jimenez (first author) · 2023

    Read the original

    arXiv 2310.06770OpenAlex W4387561453

    On the reading lists

    Why it matters

    Established realistic repository-level software engineering as a benchmark for autonomous coding agents.

    Condensed from the expert lists' notes.

    The record

    First released
    2023 (from arXiv 2310.06770)
    Citations
    4,406 citations, as counted by Papers With Code (CC BY-SA 4.0)
    Named by
    the chatgpt list only
    Score
    0.338 · computed rank 510 · consensus 0.3, must-read 0, citations percentile 0.708 (2020s · language), awards 0, existing lists 0. Method

    Data sources

    Catalogue data from OpenAlex (CC0), Crossref (CC0 metadata; no abstracts are shown), arXiv, and Papers With Code (CC BY-SA 4.0). Each count names its source. See the method.