Gradient-Based Learning Applied to Document Recognition

Inside the paper

The problem

Recognizing handwritten documents involves more than identifying isolated characters: systems must also handle uncertain segmentation and context. Traditionally, designers assembled separate modules and hand-built features or rules for parts of this process.

The idea

The paper trains convolutional networks to recognize character images and combines recognition modules in graph transformer networks. A whole-system objective can then guide the trainable components together.

How it works

• A convolutional network processes pixel images using local receptive fields and shared weights, building features through successive layers.
• For a character string, a segmentation module can propose multiple candidate cuts instead of committing immediately to one interpretation.
• Graph transformers represent and combine alternative segments, character labels and language constraints as paths with associated scores.
• A global loss evaluates the resulting interpretations, and gradients train the participating modules to improve the overall decision.

What it showed

The authors compare digit recognition methods on the modified NIST dataset and report strong results for convolutional networks. They also report improvements from global training in handwriting recognition and describe a deployed check-reading system.

Read it for

Start with Section II and the LeNet-5 diagram for the image model, then Section III for the digit comparisons. Sections IV and VI explain graph transformers and global training; the later system details can be skimmed until you want an application.

Written in our own words from the paper itself. A draft the editors have not yet reviewed.