Neural networks and physical systems with emergent collective computational abilities

Inside the paper

The problem

A useful memory should be able to recover an item from an incomplete or imperfect cue. The paper asks whether that behavior can arise from the collective dynamics of many simple connected units.

The idea

Hopfield describes memories as stable states of a recurrent network. Starting from a partial or corrupted pattern, the network changes its state until it settles in a nearby stable configuration.

How it works

• Each unit has a binary state and receives weighted input from the other units in the network.
• Units update asynchronously, switching according to whether their summed input exceeds a threshold.
• Set connection strengths using the patterns to be stored, so those patterns tend to become stable states of the system.
• For symmetric connections, an energy function decreases as units change state, giving a way to understand convergence to local minima.

What it showed

The paper analyzes the dynamics and reports simulations in which starting states near stored patterns often settle on those memories. It also observes limits: other stable states can occur, and retrieval becomes less reliable as stored patterns compete.

Read it for

Start with the discussion of content-addressable memory on paper page 2555, then follow the storage rule and energy argument on pages 2555–2556. The simulation results and Figure 1 on page 2557 give a concrete view of retrieval.

Written in our own words from the paper itself. A draft the editors have not yet reviewed.