A Simple Framework for Contrastive Learning of Visual Representations

Inside the paper

The problem

Learning image representations without labels offered a way to use large collections of images, but the role of different training choices remained difficult to isolate across existing contrastive methods.

The idea

SimCLR learns to recognize two altered views of the same image as a pair. The authors use a shared framework to investigate which transformations and training components make the resulting representations useful.

How it works

• Two independently sampled sequences of image transformations produce related views of each training image.
• A convolutional encoder maps each view to a representation, and a small nonlinear projection network maps it again for the training objective.
• The contrastive loss brings representations of matching views closer while distinguishing them from views of other images in the batch.
• After pretraining, the projection network is discarded and the encoder representation can be evaluated using a linear classifier or fine-tuned with labels.

What it showed

The experiments show that combinations of augmentations, a nonlinear projection, and larger batches affect the quality of learned representations. A linear classifier on the best reported ImageNet representation reaches 76.5% top-1 accuracy.

Read it for

Figure 2 and Section 2 describe the learning task, and Section 3 isolates its components. Start with the augmentation and projection comparisons before skimming the training setup in Section 2.2.

Written in our own words from the paper itself. A draft the editors have not yet reviewed.