%%\documentstyle[twocolumn,epsf,IEEEtran]{article}
%%\documentstyle[twocolumn,epsf]{IEEEtran}
\documentstyle[twocolumn,eclepsf]{IEEEtran}

\def\bs{-0.5ex}
\newtheorem{theorem}{Theorem}

\begin{document}

\title{Application of an Evolution Strategy \\to the Hopfield Model of Associative Memory}

\author{Akira Imada, Keijiro Araki\thanks{A. Imada is with the
Graduate School of Information Science, Nara Institute of Science and Technology, 
Nara, Japan. E-mail: akira-i@is.aist-nara.ac.jp .}\thanks{K. Araki is with the 
Graduate School of Information Science and Electrical Engineering, Kyusyu University,
Fukuoka, Japan. E-mail: araki@c.csce.kyusyu-u.ac.jp .}}

\pagestyle{empty}
\maketitle

\begin{abstract}
\noindent
We apply evolutionary computations to Hopfield's neural network model of associative memory.
In the Hopfield model, almost infinite number of combinations of synaptic weights give a  network a function of associative memory.
Furthermore, there is a trade-off between the storage capacity and size of basin of attraction.
Therefore, the model can be thought of as a test suite of multi-modal and/or multi-objective function optimization.
As preliminary stages, we investigate the basic behaviors of associative memory under simple evolutionary processes.
In this paper, we present some experiments using an evolution strategy.
\end{abstract}

\section{Introduction}
\noindent
There have been a lot of researches which apply evolutionary techniques to layered neural networks
(see e.g.,~\cite{whit,yao}, and references quoted therein).
However, their applications to fully-connected recurrent neural networks remain few so far.
In this paper we present some basic behaviors of the Hopfield model of associative memory under an evolution strategy.\\

\noindent
Associative memory is a dynamical system which has a number of stable states with a domain of attraction around them~\cite{koml}.
Applying a vector 
\( \mbox{\boldmath $x$}_{0} \) 
as the initial state of the dynamical system results in the state trajectory
\( \{ \mbox{\boldmath $x$}_{t} \}_{t \ge 0} \).
If we start the system with 
\( \mbox{\boldmath $x$}_{0}=\mbox{\boldmath $u$} + \mbox{\boldmath $\delta u$} \) 
in the vicinity of some fixed point \( \mbox{\boldmath $u$} \), 
and the state trajectory converges to \( \mbox{\boldmath $u$} \), i.e., 
\( \lim_{t \to \infty} \mbox{\boldmath $x$}_{t} = \mbox{\boldmath $u$} \),
then we can regard the system as an associative memory.\\

\noindent
In 1982, Hopfield proposed his fully connected neural network model of associative memory~\cite{hopf}.
The model consists of $N$ neurons and $N^2$ synapses.
Each neuron can be in one of two states $\pm 1$, and $p$ bipolar patterns 
%%\begin{displaymath}
$
{\xi}^{\nu} = ({\xi}^{\nu}_{1},\cdots,{\xi}^{\nu}_{N}) \hspace{.1in}({\nu} = 1, \cdots, p)
$
%%\end{displaymath}
are memorized as equilibrium states.
Hopfield employed a discrete-time, asynchronous update scheme.
Namely, each neuron updates its state, one at a time, according to 
%%the sign of the weighted sum of of the states of all other neurons, i.e.,
\begin{displaymath}
%%$
s_{i}\hspace{.02in} (t+1) = sgn \hspace{.02in} \left( \hspace{.04in} \sum_{j \ne i}^N { w_{ij} s_{j}(t)} \hspace{.04in} \right),
%%$
\end{displaymath}
where $s_{i}(t)$ is a state of $i$-th neuron at time $t$, $w_{ij}$ is a synaptic weight from neuron $j$ to neuron $i$.\\
%%where $s_{i}(t)$ is a state of $i$-th neuron at time $t$, $w_{ij}$ is a synaptic weight from neuron $j$ to neuron $i$, 
%%and $f(z) = 1$ if $z \ge 0$ and $-1$ otherwise. \\

\noindent
%%Asynchronous update means that at most one neuron is updated its state at each time step.
The behaviors of the collective states of the individual neurons are characterized by the synaptic weights.
%%the $N^{2}$ synaptic weights $w_{ij}$.
Networks store some number of patterns as fixed points when these synaptic weights are determined appropriately.
Hopfield specified these $w_{ij}$'s by the Hebbian rule~\cite{hebb}, i.e.,
\begin{displaymath}
%%$
w_{ij} = \frac{1}{N} \sum _{ \nu = 1}^{p} {{ \xi }_{i}^{ \nu }{ \xi }_{j}^{ \nu }}\hspace{.1in}(i \ne j), \hspace{.02in} w_{ii} = 0.
%%$
\end{displaymath}
Then giving one of the memorized patterns, possibly including a few errors, to the network as an initial state
will result in the stable state after certain time steps.
%% of update.
Hopfield suggested by using computer simulation that the maximum number of patterns stored in a network with $N$ neurons is $0.15N$,
on the condition that a small error is allowed in recalling.
%%if a small error in recalling is allowed.
Later, this was theoretically calculated by Amit et al.~\cite{amit} by using {\it replica} method.
They showed the storage capacity is $ 0.138N $.
%%Amit, Gutfreund \& Sompolinsky\cite{amit}
In 1987, McEliece et al.~\cite{mcel} proved that when
%%McEliece, Posner, Rodemick \& Venkatesh\cite{mcel}
%%\begin{displaymath}
$
p < N / 4 \ln N
%%p < \frac {N}{4 \ln{N}}
$
%%\end{displaymath}
holds, the Hopfield's model will recall the memory without error.\\

\noindent
The storage capacity depends strongly on how synaptic weights are specified. 
The specification of the synaptic weights is conventionally referred to as {\it learning}.
Other learning schemes instead of the Hebbian rule have been proposed for their increased storage capacity. 
The {\it pseudo-inverse matrix method} by Kohonen et al.~\cite{koho}
and the {\it spectral algorithm} by Pancha et al.~\cite{panc} are the examples.
The capacity is enlarged to $p=N$.
%%by Kohonen \& Ruohonen\cite{koho}
%%{\it spectral algorithm} by Pancha \& Venkatesh\cite{panc}
Both are extensions of the Hebbian learning.\\

%The specification of synaptic weights also implies that pruning the connection of Hebbian weights.
%%Derrida et al.~\cite{derr} studied a symmetrically diluted Hebbian synaptic matrices and showed that 
%%the capacity, measured per surviving synapse, is significantly higher than that of the fully connected network.
\noindent
Then, what is the ultimate capacity when only the learning scheme is explored within the Hopfield's framework?
Generally the storage capacity is traded off against basin size.
Gardner~\cite{gard} studied the optimal storage capacity as a function of the size of basin of attraction.
She showed that the ultimate capacity will be $p=2N$ as basin size tends to zero, 
and also proposed  an algorithm to realize it.\\

\noindent
The storage capacity also depends on the other aspects of neural networks.
%%For example, we can increase the storage capacity exponentially instead of proportionally by modifying the architecture of the network.
For example, we can increase the capacity {\it exponentially} instead of {\it proportionally} by modifying the architecture of the network.
%%Chiueh \& Goodman\cite{chiu}
Chiueh et al.~\cite{chiu} proposed a new architecture in which 
storage capacity scales {\it exponentially} with $N$, the number of neurons.
They showed that $2^N$ might be the ultimate upper bound for the capacity.\\

\noindent
However, our interest in dealing with the neural network model of associative memory 
is not to pursue the higher storage capacity, 
but to use it as a test function of evolutionary computations. 
The Hopfield model has some favorable properties as a test function of continuous parameter optimization problems.
First of all, the model is very simple and well documented.
Second, for a small number $p$ of given patterns to be stored, 
there exist almost infinite number of combinations of synaptic weights which give a network a function of associative memory (multi-modality).
As $p$ increases, the number of solutions becomes small, and finally it will be a needle-in-a-haystack-problem to locate the 
global optimum in the sense of Gardner~\cite{gard}.
Third, there is a trade-off between the storage capacity and error correcting capability. 
Hence the model has a property of multi-objectivity.
However, we have much fewer  knowledge about the solutions than the usually used test functions.
Therefore we believe that these studies shed some light on the new analysis of the Hopfield model as well.\\

\noindent
Previously, with using a genetic algorithm~\cite{holl,gold},
we successfully evolved a fully-connected neural network with random synaptic weights
to store some number of patterns~\cite{imad-1}.
It was made by optimally pruning some of its connections 
as well as balancing the number of excitatory/inhibitory synapses.
Chromosomes consisting of mostly 1 but a few of 0 and -1
were used to modify a pre-determined random matrix $R_{ij}$, i.e.,
individuals in a population were modified with equation
\begin{displaymath}
R_{ij} \times c_{ij},
\end{displaymath}
where $c_{ij} \in \{1, 0, -1\}$.
The storage capacity obtained was $0.16N.$\\

\noindent
We also showed that the genetic algorithm enlarges the capacity of Hebb-rule associative memory~\cite{imad-2},
where we used an over-loaded Hebbian weight matrix $J_{ij}$,
which means that the matrix learned a set of patterns exceeding the storage capacity ($p > 0.15 N$).
%%That is, the number of learened patterns by Hebbian rule exceeded the capacity 
This matrix 
%%\begin{displaymath}
%% w_{ij} = c_{ij} \times J_{ij}, \hspace{.08in}  c_{ij} \in \{1, 0, -1\}
%%\end{displaymath}
was also evolved in the same way as above, 
and eventually stored all the learned patterns as fixed points.
The capacity $0.33N$ was obtained.\\

\noindent
In the genetic algorithm above, we used ternary chromosomes whose alleles modify synaptic weights.
In this paper, on the other hand, 
synaptic weights are directly encoded into chromosomes,
and applied an Evolution Strategy~\cite{rech}.
There the synaptic weights $W_{ij}$'s are modified by small real variables $\eta_{ij}$'s as 
\begin{displaymath}
 W_{ij} + \eta_{ij}.
\end{displaymath}

\noindent
In the next section, we describe our evolution strategy implementation briefly.\\

\section{An evolution of the Hopfield Model}
\noindent
In this simulation, a set of $p$ random bipolar patterns ${\xi}^{\nu}$ (${\nu} = 1, 2, \cdots, p$) are produced before each run.
The goal of the evolution strategy is to search weight matrices which store all of these patterns as fixed points.
At the beginning of a run, a population of chromosomes are initialized.
In each generation, these chromosomes are modified through crossover and mutation operation,
and fitness value of individual phenotype is evaluated.
According to the fitness values, individuals of the next generation are selected,
using a ($\mu + \lambda$)-strategy.
The cycle of reconstructing the new population with better individuals and restarting the search is repeated until a perfect
solution is found or a maximum number of generation has been reached.\\

\noindent
The conceptual algorithm is as follows:\\
%\begin{small}

\noindent
\hspace*{0in}{\bf begin}\\
\hspace*{0.15in} {\it initialize} 256 individuals\\
\hspace*{0.15in} {\it evaluate} fitness values of the individuals\\
\hspace*{0.15in} {\bf while} termination criterion not fulfilled {\bf do}\\
\hspace*{0.25in} {\bf until} the rest of the best 40\% replaced {\bf do}\\
\hspace*{0.35in} {\it select} two parents from the best 40\% individuals\\
\hspace*{0.35in} {\it recombine} them to produce an offspring\\
\hspace*{0.35in} {\it mutate} it\\
\hspace*{0.35in} {\it evaluate} its fitness value\\
\hspace*{0.25in} {\bf od}\\
\hspace*{0.15in} {\bf od}\\
\hspace*{0in}{\bf end}\\
%\end{small}


\noindent
We describe each procedure a bit more in detail.\\

\begin{description}
\item[(1)]{\bf (initialization)}
Individuals are $2N^2$-dimensional vectors whose components are $N^2$ synaptic weights (object variables)
and the same number of standard deviations (strategy parameters).
Although individuals are usually initialized at random, 
in this paper, the object variables are initialized with an identical matrix, i.e., either
(a) all zero synaptic weights or (b) over-loaded Hebbian weights.
The former case is referred to as {\it building memory on a tabula rasa}.
In both cases, networks in the first generation do not function as associative memories.
Population number is set to 128.
\item[(2)] {\bf (fitness evaluation)}
When one of memorized patterns ${\xi}^{\nu}$ is given to the network as an initial state, 
the state of neurons varies from time to time afterwards (unless the ${\xi}^{\nu}$ is a fixed point).
In order for the network to function as associative memory, the instantaneous network state must be similar to the input pattern. 
The similarity is defined by,
\begin{displaymath}
m^{\nu}(t) = \frac{1}{N} \sum_{i=1}^{N} { {\xi}_{i}^{\nu} s^{\nu}_{i}(t) },
\end{displaymath}
where $s_{i}^{\nu}(t)$ is the state of neuron $i$ at time $t$ for the input of $\xi^{\nu}$.
%%Clearly it is related to Hamming distance between the two states.
These $m^{\nu}(t)$'s are referred to as {\it overlaps}.
The quality of retrieval of memorized patterns are represented by 
each temporal average of $m^{\nu}(t)$'s over certain time interval $t_0$. 
These $p$ parameters are denoted by $\left \langle m^{\nu} \right \rangle$.
We evaluate the fitness value of each network by further averaging these $\left \langle m^{\nu} \right \rangle$'s over all memorized patterns. 
Namely, the fitness $f$ is
\begin{displaymath}
f = \frac {1}{t_{0} \cdot p}
\sum_{t=1}^{t_{0}}{\sum _{\nu=1}^{p}{m^{\nu}(t)} }.
\end{displaymath}
In this paper, $t_{0}$ is set to $2N$, twice the number of neurons.
Note that the fitness 1 implies all the $p$ patterns are stored as fixed points, while all other cases have a fitness less than 1.
\item[(3)] {\bf (selection)}
We use ($\mu + \lambda$)-selection.
Two parent individuals are chosen {\it randomly} from the best 40\% of the population. 
%% (mu + lambda) - selection 
\item[(4)] {\bf (recombination)}
{\it Discrete recombination}, which is similar to {\it uniform crossover}~\cite{sysw} in binary chromosomes, 
are operated both on object variables and strategy parameters of the selected parents, i.e.,
two parents $(u_1, \cdots, u_n)$ and  $(v_1, \cdots, v_n)$ produce an offspring $(w_1, \cdots, w_n)$
such that $w_i$ is either $u_i$ or $v_i$ with equal probability.
%% kokoni discrete recombination no teigi
This process is repeated until all the individuals in the worst 60\% of the population are replaced.
\item[(5)] {\bf (mutation)}
We also apply {\it evolutionary programming}~\cite{foge} to the Hopfield  model of  associative memory~\cite{imad-3}.
For comparison purposes, we use the similar mutation operation to that implementation.
That is to say, 
instead of usual mutation in evolution strategy:
\begin{displaymath}
\left\{
\begin{array}{ll}
x_{i}' = x_{i} + {\sigma_{i}'}\\
{\sigma}_{i}' = {\sigma}_{i} + \exp{ N_{i}(0, 1)},\\
\end{array} \right.
\end{displaymath}
we mutate object variable $x_{i}$ and strategy parameter ${\sigma}_{i}$ as follows:
\begin{displaymath}
\left\{
\begin{array}{ll}
x_{i}' = x_{i} + \sqrt{ f_{i} } \cdot N_{i}(0, 1)\\
{\sigma}_{i}' = {\sigma}_{i} + \sqrt{ 0.01 \cdot f_{i} } \cdot N_{i}(0, 1),
\end{array} \right.
\end{displaymath}
where $N_{i}(0, 1)$ is normally distributed random variable of mean 0 and standard deviation 1 sampled for each individual $i$.\\
\end{description}

\noindent
The above parameter values were determined mainly on the basis of trial and error.
Note that they have not optimally been tuned.

\section{Results and Discussion}
\noindent
All simulations were carried out on networks with 49 neurons.
Hence, the first goal of the experiment is to find a solution out of many possible solutions
distributed in the huge synaptic weight space of \( \mbox{\boldmath $R$}^{2401} \). \\

\noindent
The effect on evolution of varying $p$ is studied.
We repeat each simulation 30 times with different random number seed.
If we find the perfect solution(s) then we increment $p$.
Thus, we found a matrix evolved to store a maximum of 11 patterns 
started with a population of zero matrices, and a maximum of 17 patterns started with over-loaded Hebbian matrices.\\

\noindent
Figure~\ref{evolution} shows representative samples of the best fitness vs. generation 
for evolutions started with both zero and over-loaded Hebbian matrices.
They converge at generation 1062 and 4374, respectively.\\

\begin{figure}[hbtp]
 \begin{center}
   \epsfile{file=fitness.eps,width=9cm,height=9cm}
   \caption{\bf Fitness of the Best of Generation}
   \label{evolution}
 \end{center}
\end{figure}

\noindent
Intuitively, these two matrices seem to belong to different classes of solutions.
To see this, we investigated the time evolution of degree-of-symmetry of these matrices, defined by
\begin{displaymath}
\sigma = \sum_{i=1}^{N} \sum_{j=1}^{N} {w_{ij} w_{ji}} \cdot \left( \sum_{i=1}^{N} \sum_{j=1}^{N} {w_{ij}^{2}} \right)^{-1},
\end{displaymath}
following after Krauth et al.~\cite{krau}.
In the simulation starting with over-loaded Hebbian matrices, 
they are totally symmetric at the  beginning.
Then the degree of symmetry decreases and asymptotically approaches to the value around 0.5, as memory states are emerged.
In this level of over-saturation, the memory states are not randomized as 
the Kirkpatrick-Sherrington model of spin-glasses~\cite{kirk},
but some  spurious equilibrium states prevent the input  from converging to its memory state.
We conjecture that the asymmetry of the synaptic weights introduced by the evolution strategy destabilizes these spurious states,
as Herz et al.~\cite{herz} suggested in their analysis using Ising model.\\

\noindent
In the simulation starting with zero matrices, on the other hand, 
they are totally asymmetric at the beginning.
Then we can see the degree of symmetry increases rapidly, and approaches to the value around 0.4,
showing that some extent of symmetry is needed to create memory states.\\

\noindent
However the role of asymmetry of the synaptic weights is still an open question.\\

\begin{figure}[hbtp]
 \begin{center}
   \epsfile{file=symmetry.eps,width=9cm,height=9cm}
   \caption{\bf Degree of Symmetry}
   \label{symmetry}
 \end{center}
\end{figure}

\noindent
To regard a network as an associative memory system, it should show the tolerance for certain amount of noises.
That is, each memory pattern must have a certain domain of basin of attraction.\\

\noindent
%%As the criterion of basin size, we used somewhat similar definition to the one by Verleysen, Legat, \& Jespers\cite{verl},
As the criterion of basin size, we used a definition by Verleysen et al.~\cite{verl}.
We generate 1000 initial input patterns by randomly picking up a stored pattern.
They are flipped $d$ bits at random.
Then they are given to the network 
and updated $2N$ times (possibly they reach a stable state). The resulting states is then compared with the corresponding stored pattern,
and if they match {\it exactly} the run is called a success.
These are repeated, incrementing the number of noisy bit $d$ until it reaches 20.\\

\noindent
In Figure \ref{basin}, the number of successes out of 1000 runs 
are plotted against the number of input errors $d$,
using two matrices obtained from above two experiments.
For comparison, we also plot the results of the Gardner's algorithm~\cite{gard} for the same number of patterns.
%since the Hebbian rule cannot specify weights to store the same amounts of patterns as in above experiments,
Although the networks evolved by the evolution strategy have extremely small size of basin of attraction,
which implies there remains a lot of spurious attractors around the memory states.\\

\begin{figure}[hbtp]
 \begin{center}
   \epsfile{file=basin.eps,width=9cm,height=9cm}
   \caption{\bf Error Correcting Capability}
   \label{basin}
 \end{center}
\end{figure}

\section{Conclusions}
\noindent
We have described an application of evolution strategy to 
the Hopfield model of associative memory.
The objective of these experiments is to find some solutions of real-valued synaptic weight matrices
which give a function of associative memory to the network.\\

\noindent
The simulations in this paper were made with 49 neurons.
%%Hence, solution is a point in the space of \( \mbox{\boldmath $R$}^{2401} \).
This size of Hebb-rule associative memory would store around 8 random bipolar patterns as fixed points.
The evolution strategy finds solution which stores 11 ($0.22 N$) 
and 17 ($0.35 N$) patterns starting with zero matrices  and the Hebbian matrices, respectively.
%%their degrees of symmetry are different from each other.
We conjecture these phenomena are more or less due to asymmetry of a weight matrix introduced by the evolution strategy.\\

\noindent
The results and analysis reported here are only the beginning of our research to apply evolution strategy to the Hopfield model of associative memory,
and many other tasks including more elaborate tuning of the parameters are still waiting for exploration.\\

\section*{Acknowledgments}
\noindent
We thank Peter Davis at Advanced Telecommunication Research Institute (ATR)
for providing us great insight into the dynamics of Hopfield neural networks.

\begin{thebibliography}{99}

\bibitem{whit}J.~D.~Schaffer, D.~Whitley, and L.~J.~Eshelman (1992)
{\em ``Combinations of Genetic Algorithms and Neural Networks: A Survey of the State of the Art.''},
Proceedings of the Workshop on Combinations of Genetic Algorithms and Neural Networks, pp1--37.

\bibitem{yao}X.~Yao, {\em ``A Review of Evolutionary Artificial Neural Networks.''} (1993)
International Journal of Intelligent Systems, vol.8, pp539--567.

\bibitem{koml}J.~Koml\'{o}s, and R. Paturi (1988)
{\em ``Convergence Results in an Associative Memory Model.''} 
Neural Networks 1, pp239--250.

\bibitem{hopf}J.~J.~Hopfield (1982) 
{\em ``Neural Networks and Physical Systems with Emergent Collective Computational Abilities.''}, 
Proceedings of the National Academy of Sciences, USA, 79, pp2554--2558.

\bibitem{hebb}D.~O.~Hebb (1949)
{\it The Organization of Behavior.}, Wiley.

\bibitem{amit}D. J. Amit, H.~Gutfreund, and H.~Sompolinsky (1985)
{\em ``Storing Infinite Number of Patterns in a Spin-glass Model of Neural Networks.''}
Phys. Rev Lett. 55,  pp1530--1533.

\bibitem{mcel} R. J. McEliced, E. C. Posner, E.~R.~Rodemick, and S.~S.~Venkatesh (1987)
{\em ``The Capacity of the Hopfield Associative Memory.''}
IEEE Trans. Information Theory IT-33, pp461-482.

\bibitem{koho} T. Kohonen, and M. Ruohonen (1973)
{\em `` Representation of Associated Data by Matrix Operators.''}
IEEE Trans. Computers C-22(7),  pp701--702.

\bibitem{panc}G. Pancha, S. S. VenKatesh (1993)
{\em ``Feature and Memory-Selective Error correction in Neural Associative Memory.''}
in M.~H.~Hassoun eds. Associative Neural Memories: Theory and Implementation,
Oxford University Press, pp225-238.

\bibitem{gard} E. Gardner (1988)
{\em ``The Phase Space of Interactions in Neural Network Models.''}
J. Phys 21A, pp257--270.

\bibitem{chiu} T. D. Chiueh, and R. M. Goodman (1991)
{\em ``Recurrent Correlation Associative Memories.''}
IEEE Trans. Neural Networks 2(2), pp275--284.

\bibitem{holl}J. Holland (1975)
{\it Adaptation in Natural and Artificial Systems.}
The University of Michigan Press

\bibitem{gold}D. Goldberg (1989)
{\it Genetic Algorithms in Search, Optimization, and Machine Learning.} 
Addison-Wesley 

\bibitem{imad-1} A.~Imada, and K.~Araki (1995)
{\em ``Mutually Connected Neural Network Can Learn Some Patterns by Means of GA.''},
Proceedings of the World Congress on Neural Networks, vol.1, pp803--806.

\bibitem{imad-2}A.~Imada, and K.~Araki (1995)
{\em ``Genetic Algorithm Enlarges the Capacity of Associative Memory.''},
Proceedings of the 6th International Conference on Genetic Algorithms, pp413--420.

\bibitem{rech} I. Rechenberg (1965)
{\em ``Cybernetic Solution path of an Experimental Problem.''}
Royal Aircraft Establishment, Library Translation, 1122 (Hants UK).

\bibitem{sysw}G. Syswerda (1989)
{\em ``Uniform Crossover in Genetic Algorithms.''}
Proceedings of the 3rd International Conference on Genetic Algorithms, pp2--9.

\bibitem{foge} D.~B.~Fogel (1992)
{\em ``An Analysis of Evolutionary Programming.''}
Proceedings of the 1st Annual Conference on Evolutionary Programming, pp43--51.

\bibitem{imad-3} A. Imada, and K. Araki (1996)
{\em ``Searching Real-Valued Synaptic Weights of Hopfield's Associative Memory using Evolutionary Programming.``}
The 6th Annual Conference on Evolutionary Programming (to appear)

\bibitem{krau}W. Krauth, J.-P. Nadal, and M.Mezard (1988)
{\em ``The Roles of Stability and Symmetry in the Dynamics of Neural Networks.''}
J. Phys. A: Math. Gen. 21, pp2995--3011.

\bibitem{kirk} S. Kirkpatrick, and D. Sherrington (1978)
{\em ``Infinite-ranged models of Spin-glasses.''} 
Phys. Rev. B17, pp4384-4403.

\bibitem{herz} J. A. Hertz, G. Grinstein, and S. A. Solla (1987)
{\em ``Irreversible Spin Glasses an Neural Networks.''}
in L. N. van Hemmen and I. Morgenstern eds. Heidelberg Colloquium on Glassy Dynamics. Lecture Notes in Physics 275 Springer-Verlag, pp538--546.

\bibitem{verl} M. Verleysen, J.-D. Legat, and P. Jespers (1993)
{\em ``Analog implementation of an Associative Memory: Learning Algorithm and VLSI Constraints.''}
in M.~H.~Hassoun eds. Associative Neural Memories: Theory and Implementation, Oxford University Press, pp265--275.

\end{thebibliography}

\end{document}
