Some Studies in Machine Learning Using the Game of Checkers

Inside the paper

The problem

A checkers program cannot feasibly search every possible continuation, so it needs a way to judge positions before the game ends. The question is whether experience can improve those judgments without hand-tuning every detail.

The idea

Samuel combines a look-ahead game program with methods that learn from play. One records and reuses evaluations of positions; another adjusts a weighted evaluation function to generalize across positions.

How it works

• The program searches possible continuations and scores resulting positions using features such as the balance of pieces and other board characteristics.
• In rote learning, stored positions and their earlier evaluations can be used when the same positions recur in later games.
• In generalization learning, the program compares a current evaluation with a look-ahead result and adjusts coefficients associated with board features.
• Versions of the program play against each other, allowing the author to observe changes in play and compare learning procedures.

What it showed

The reported games show improvement in particular settings, alongside difficulties such as instability in the learned evaluation. Samuel discusses limits of the experiments and the differences between memorizing positions and learning a more general scoring rule.

Read it for

Read the basic playing program on pages 536–539, then the learning discussions on pages 540–546. The sample games and feature definitions in the appendices are useful for a closer audit but can be skimmed on a first reading.

Written in our own words from the paper itself. A draft the editors have not yet reviewed.