(Brest State Technical University 2005 Spring Semester: Corse Practice)

Associative Memory by Hopfield Neural Network

Akira Imada (e-mail akira@bstu.by)

This document is still under construction and was lastly modified on June 6, 2008

INDEX 1st Week

Start with random pattern with random weight values.

2nd Week Store one pattern.

3rd Week Store multiple patterns.

4th Week

How many patterns can be stored?

1 1st week

The first task of this practice is to create your own hopfield network as follows, today. • Number of neuron N = 100

• At time t, neuron j takes a state S_{i}(t) = 1 or −1 (i = 0, 1, 2, ⋯, 99). – Time here is descrete, that is, (t = 0, 1, 2, ⋯).

• All the neuron inclucing itself connectted with each other.

– Hence total connections are N^{2} = 10, 000

• Weight value from neuron i to j denoted as w_{ji} takes a random value from −1.0 to 1.0, today.

Figure 1:

The state of neuron at time t+1 will be ON (+1) if all the weighted sum from all neurons (the other 99 neurons and itself) is positive, otherwise OFF (−1). That is,

S_{i}(t + 1) = sgn(∑_{j=0}^{99} w_{ij}S_{i}(t))  (1)

Excersize 1 On your monitor screen, display 100 neuron each of which takes a state of −1 or 1 at random. This is the pattern at t = 0. Then renew states from one time to the next and display states of 100 neurons.

2 2nd week

In the hopfield network created in the 1st week, we try to store a pttern. In the 1st week both state configuration and weight values are random, but this week we specify them.

The pattern is constructed as 2-dimensional array of N = 100 neurons like below.

Figure 2:

We interpret it as 2-dimensional pixels, not color, not gray-scale, but black-and-white ones. That is, The state of 1 is black and the state of −1 is white. Hence, today w_{ij} = w_{ji}.

Weight value is determined such that if and only if the both ends of the connection is 1, the weight value of the connection is 1, otherwise −1. The weight value of the conection from one neuron to itself are set to 0, that is, w_{ii} = 0, today. That is,

Figure 3: An example of specifying weight values. The value is +1, −1 or 0 depending on states of neurons of the both end of the connection.

Or, if you like mathematics, the fomula of determining w_{ij} is

w_{ij} = ξ_{i}ξ_{j} (i ≠ j), w_{ii} = 0  (2)

where ξ_{i} is the state of i-th neuron.

Excersize 2 On your monitor screen, display 100 neuron each of which takes a state of −1 or 1 such that those neurons whose state 1 construct a border of rectangle while neurons of −1 are backgraound. Specify weight of connections using Eq. (2). Start with some noisy pattern try to observe at time is 1, 2, · · ·.