Attention Is All You Need
The architecture that changed how we build language models. A closer look at attention, parallelism, and what makes a Transformer work.
Vaswani et al.
The open catalogue
A starting point for understanding AI. Explore the ideas, examine the evidence, and make up your own mind.
8 papers in the starter collection
The architecture that changed how we build language models. A closer look at attention, parallelism, and what makes a Transformer work.
From noise to structure. Understand the surprisingly simple idea behind a generation of image models.
How model size, data, and compute shape performance—and why the assumptions behind a scaling curve matter.
Learning representations from both directions of a sentence, then adapting them to language understanding tasks.
An exploration of what a large language model can learn from a handful of examples in its context.
Connecting images and language through contrastive learning, with a path to zero-shot visual recognition.
A landmark approach to predicting three-dimensional protein structure from sequence information.
Treat an image as a sequence of patches, and revisit the role of convolution in visual recognition.
This is the beginning of the library.
Help us decide what belongs here next.