the open
    research
    Contribute

    A home for curious minds

    Research,
    in the open.

    Discover the papers that matter. Question the claims. Share what you learn. Better AI research starts with an open conversation.

    Explore the research Find your starting point

    For the people who read beyond the abstract.

    THE OPEN RESEARCH / READING ROOM
    arXiv 1706.037622017

    Attention Is All You Need

    Vaswani et al.

    The claim Replace recurrence with attention to let each token draw context from the whole sequence.

    Figure 1One query, every key.
    Review · open on five dimensions
    • Rigor
    • Reproducibility
    • Implementation
    • Transparency
    • Usefulness
    Open the paper
    A question worth asking

    Can we reproduce
    what we read?

    Not yet assessed
    • Reference code
    • Training config
    • Your reproduction
    Read. Question. Reproduce.
    Ideas worth understanding Conversations with substance Evidence you can inspectOpen by principle. Curious by nature.

    The reading room

    Worth a closer look.

    View the catalogue
    Language models2017

    Attention Is All You Need

    The architecture that changed how we build language models. A closer look at attention, parallelism, and what makes a Transformer work.

    Vaswani et al.Reviews coming soon
    Training & compute2020

    Scaling Laws for Neural Language Models

    How model size, data, and compute shape performance—and why the assumptions behind a scaling curve matter.

    Kaplan et al.Reviews coming soon

    A starter collection of real papers. Community reviews and discussions are not live yet.

    A little context goes a long way

    Big ideas.
    A place to begin.

    The field moves fast. Its foundations are worth slowing down for. Follow a reading path through the ideas that shaped modern AI.

    Explore the essentials Growing toward a curated collection of 200–500 papers.
    A reading path

    The language
    of machines.

    Attention · BERT · GPT-303 papers
    A reading path

    Learning
    to see.

    CLIP · Vision Transformer02 papers

    Trust is built in the details

    More than
    a single score.

    A bold claim and a working implementation are different things. Our review framework makes room for both—and the evidence behind each.

    How we think about reviews
    The review frameworkProposed dimensions

    Do the methods, baselines, and experiments support the claims?

    Reasoned reviews. Visible evidence. Room to disagree.

    Read something interesting?

    Bring it to the table.

    Contribute a paper