\documentstyle[seceqn,twocolumn,10pt,eclepsf]{baltzer}

\begin{document}
\begin{frontmatter}  
\title{A Genetic Algorithm using an Entropy based Selection Scheme}
\author{Akira Imada}
\address{Nara Institute of Science and Technology\\
         1-11-13-604 Abeno, Hannan, Osaka 545-0021 Japan
         \email{ai@kk.iij4u.or.jp}}

\begin{abstract}
\noindent
In genetic algorithms, an evolution proceeds by selecting some individuals in one generation 
to construct the next generation.
Most of the selection schemes so far proposed have been based on fitness value of each individual,
such as the fitness proportionate selection.
Lee and Han, on the other hand, proposed a selection scheme based on entropy of the population.
In this paper, we test the Lee and Han's method using the Hopfield model of associative memory as a test function.
\end{abstract}

\begin{keywords}
genetic algorithm, entropic sampling, Hopfield model of associative memory
\end{keywords}
\end{frontmatter}

\section{Introduction}
\noindent
Genetic algorithms usually employ selection schemes based on fitness value of each individual in the population, 
which sometimes makes a search very difficult especially in situations like the so-called needle in haystack problems.
In this paper, we try another selection scheme which is based on entropy of the population.
To be more specific, offspring produced is more likely to survive if it decreases the entropy of the current population. 
That is, the selection rate of rare individuals is higher than that of abundant individuals.
We use here the fairly well studied Hopfield model of associative memory as a test function.

Associative memory is a dynamical system 
which has a number of stable states with a domain of attraction around them \cite{koml}.
If the system starts at any state in the domain, it will converge to the stable state.
Hopfield \cite{hopf} proposed a fully connected neural network model of associative memory
in which information is stored by being distributed among neurons, and
we can retrieve the information from dynamically relaxed neurons' states.
In the model, some of the appropriate configurations 
of synaptic weights give the network a function of associative memory.
Namely, the network stores a {\it fixed} set of bipolar patterns, sequences of $+1$ and $-1$, as attractors.
Thus, the capability to store a set of patterns defines a fitness landscape on {\it synaptic weight space},
or equivalently, a hyper-plane defined on all the possible configurations of weight values of a network.
In contrast, a {\it fixed} configuration of synaptic weights also defines fitness landscape on {\it pattern space}.
For example, the weight configuration prescribed by the learning algorithm proposed by Hebb \cite{hebb} 
to store a set of given patterns 
creates a fitness landscape which has the {\it global} peaks at the locations of given patterns 
as well as many other {\it local} peaks.
In this paper, we explore the landscape constructed by this fixed configuration of the Hebbian synaptic weights 
on bipolar pattern space.
The reason why we chose pattern space here instead of weight space is 
that weight space is much more difficult to explore than pattern space
due to its high dimensionality and real-valued independent variables.
The goal of this study, however, is the exploration of weight space 
to find all the solutions in weight space which is still an open issue.

The algorithm we employ in this paper was originally proposed by Lee and Han \cite{lee}.
They successfully applied the algorithm to the Kauffman's NK-model \cite{kauf}.
They noted the fitness dependent sampling fails to search for solutions
when there exist large barriers between solutions like the NK-model.\\

\noindent
The remainder of this paper is organized as follows. 
Since we explore a fitness landscape by a GA,
some remarks on the concept of fitness landscape is briefly given in Section 2.
Section 3 gives some descriptions about the Hopfield model and how the fitness is evaluated there.
The algorithm employed is described in detail in Section 4. 
After presenting experimental results and discussion in Section 5, we present concluding remarks.

\section{Fitness Landscape}
\noindent
The concept of the fitness landscape was first introduced by Wright \cite{wrig} in 1932,
to study biological evolutionary processes. 
Since then, this concept has been used not only in evolutionary biology 
but also in chemistry, physics, computer science and so on.

In chemistry, for example, a molecule can be represented as a string 
of $N$ letters with each letter being chosen from an alphabet of size $k$ \cite{mack}.
Twenty amino acids ($k=20$) for proteins or four nucleotides ($k=4$) 
for nucleic acids can be considered as examples of the alphabet.
The $k^N$ possible combinations of the letters construct a configuration space of the string.
Then, for example, the free energy of RNA folding into secondary structures \cite{font},
or the ability of peptides to bind to a particular substrate to catalyze a specific reaction \cite{mayn}
is assigned as a fitness value to each configuration.
In physics, the Hamiltonian energy of Ising spins defines a fitness landscape on the configuration space of $N$ spins.
Each spin takes the value either 1 or $-1$ ($k=2$).
Bray and Moore \cite{bray} argued about the number and distribution of meta-stable states (local optima) 
of the Hamiltonian energies.

Our fitness landscape defined in this paper is somewhat similar to this Bray and Moore's Hamiltonian energy surface.
Their Ising spins correspond to our neurons' states and the stable states of the Hamiltonian energies
roughly imply the memory states of our associative memory network.
That is, the fitness landscape here are defined 
on all the possible configuration of $N$-bit bipolar patterns by evaluating
their fitness value, i.e.,  
the degree to how each of these patterns is stored in the network as a fixed point attractor.
This fitness evaluation is described in the next section.

To explore these fitness landscapes, we need a rule by which a point in the space moves to one of its neighbors.
Then, consecutive movements of a point to the neighbors form a {\it walk} on the landscape.
Macken et al.\ \cite{mack} used {\it random point mutation} 
that changes a single letter in the string to specify neighbors of the string.
Then, by sampling points along an {\it evolutionary walk} 
in which point moves to the {\it firstly found fitter} neighbor,
they  studied the statistical properties of the landscape of the chemical 
affinity of antibody for  antigen in immune response. 
Weinberger \cite{wein} used two different walks: ``gradient walk'' 
in which the walker steps to the {\it best} of its neighbors and {\it ``random adaptive walk''} 
in which the next step is chosen {\it at random} from the set of better neighbors,
to investigate the Kauffman's NK landscape which is a model formulated in more general form \cite{kauf}.

Our explorations are made by giving the points on pattern space 
the two evolutionary operations, mutation and crossover, which is described in Section \ref{ga}.

To conclude this section, we must note the following remarks.
The fitness landscapes described above are all constructed on the space of {\it discrete} configurations of string.
What we are concerned, however, is a fitness landscape defined on the {\it continuous} weight space, 
following Gardner \cite{gard} who discussed solutions in terms of the volume of the solutions in weight space.
The number and distribution of the solutions are still open problems and
one of our goals of this study is to obtain the information. 
This paper is one of the preliminary stages for the purpose.

\section{Hopfield Model}
\noindent
The Hopfield model consists of $N$ neurons and $N^2$ synapses.
Each neuron state is either active (+1) or quiescent ($-1$).
When an arbitrary $N$-bit {\it bipolar pattern} is given to the network as an initial state,
the dynamical behavior of neurons' states afterwards are characterized by the strengths of the $N^2$ synapses.
The synaptic strengths are called {\it weights}, and the weight from neuron $j$ to neuron $i$ is denoted 
as $w_{ij}$ in this paper.
Provided the synaptic weights are determined appropriately, network can store some number of patterns as attractors.
Hopfield employed the so-called Hebbian rule \cite{hebb} to prescribe the weights.
That is, to store $p$ bipolar patterns $\xi^{\nu}$:
\begin{displaymath} 
{\xi}^{\nu} = ({\xi}^{\nu}_{1},\cdots,{\xi}^{\nu}_{N}), \hspace{.2in}{\nu} = 1, \cdots, p,
\end{displaymath}
the weight values are determined as:
\begin{displaymath}
w_{ij} = \frac{1}{N} \sum _{ \nu = 1}^{p} {{ \xi }_{i}^{ \nu }{ \xi }_{j}^{ \nu }}
           \hspace{.1in}(i \ne j), \hspace{.1in} w_{ii} = 0.
\end{displaymath}
An instantaneous state of a neuron is updated asynchronously (one neuron at a time) as:
\begin{displaymath}
s_{i}\hspace{.02in} (t+1) = sgn \hspace{.02in} \left( \hspace{.04in} \sum_{j \ne i}^N { w_{ij} s_{j}(t)} 
     \hspace{.04in} \right),
\end{displaymath}
where $s_{i}(t)$ is a state of the $i$-th neuron at time $t$.
If an initial state converges to one of the stored patterns $\xi^{\nu}$ 
as an equilibrium state, then the pattern is said to be {\it recalled}.
Furthermore, if an initial state chosen from the stored patterns remains unchanged from the beginning,
then the pattern is said to be stored as a {\it fixed point}.

In analyzing the Hopfield model, 
there have been basically two different approaches:
one is to explore {\it pattern space} searching for attractors under a specific weight configuration, and 
the other is to explore {\it weight space} searching for an appropriate weight configuration
that stores a given set of patterns. 
To be more specific, the former is an analysis of the Hamiltonian energy 
as a function of all the possible configurations of bipolar pattern to be given to the network,
where the synaptic weights are pre-specified using a learning algorithm, usually the Hebb's rule, 
so that the network stores a set of $p$ given patterns.
In this context, the model for $p=1$ corresponds to the Mattis model of spin-glass \cite{matt},
in  which the Hamiltonian energy has two minima,
while the model for infinitely large $p$ corresponds to the Sherrington-Kirkpatrick model \cite{SK}, 
in which the synaptic weights become  Gaussian random variables.
Analyses of the former type have been made in between these two extreme cases (see Amit \cite{amit}).
The latter analysis, on the other hand, was addressed by Gardner \cite{gard}.
She discussed the optimal weight configurations for a {\it fixed} number of given patterns
in terms of the volume of the solutions in weight space, 
suggesting that the volume shrinks to vanish when $p$ approaches to $2N$.
In short, the former approach searches 
for the optimal pattern configurations which minimize the Hamiltonian energy 
{\it in pattern space} {\it with the weights being fixed},
while the latter searches for the weight configurations {\it in weight space} 
that optimally store a set of given {\it fixed patterns}.

So far, we have studied the model with the latter approach.
We have explored  fitness landscapes of the model {\it defined on weight space}, and 
have found many solutions that store more patterns 
or store them with larger basin of attractions than, e.g., the Hebbian synaptic weights \cite{imad-1,imad-12,imad-5}.
Now, our interest is on the number and distribution of these solutions over the whole weight space, 
which is still an open problem.
We suspect the GA using entropic sampling is one of the appropriate tools to pursue these problems.
However, since the $N^2$-dimensional {\it continuous} weight space is much more difficult to wander around 
than the $N$-dimensional {\it discrete} pattern space,
we explore here the {\it pattern space}
to see preliminary how our fitness function works under the entropic sampling.
In other words, we use the model as a test function of the entropic sampling technique
in the sense that all solutions are {\it a priori} known, 
like in other studies using pure mathematical test functions.

Since we explore the fitness landscape defined on pattern space in this paper,
the fitness of an arbitrary configuration of $N$-bit bipolar pattern should be defined.
Here we evaluate the fitness of a pattern according to 
how the instantaneous neurons' states after the pattern is given to the network,
are similar to either of the stored patterns.
The similarity as a function of time is defined as:
\begin{displaymath}
m^{\nu}(t) = \frac{1}{N} \sum_{i=1}^{N} { {\xi}_{i}^{\nu} s^{\nu}_{i}(t) }.
\end{displaymath}
The temporal average of $m^\mu(t)$ is calculated for each stored pattern, 
and the maximum value among them is assigned to the input pattern as fitness.
Thus, the fitness value $f$ is evaluated as follows:
\begin{displaymath}
f = \max \{ \hspace{.05in} \frac {1}{t_{0}} \sum_{t=1}^{t_{0}}{m^{\nu}(t)}\hspace{.1in} \vert 
            \hspace{.1in}\mu = 1, 2, \cdots, p \}.
\end{displaymath}
In this paper, $t_{0}$ is set to $2N$, twice the number of neurons.
Note that the fitness~1 implies the pattern is a fixed point attractor,
while fitness less than~1 includes many other cases.

\section{GA Implementation} \label{ga}
\noindent
In the entropy based selection scheme,
some of the offspring produced are selected to survive to the next generation
according to the density at the fitness level to which each offspring belongs.
Here, to know the density of a pattern whose fitness is $E$, $\wp$ patterns are randomly sampled, and
the fitness value $E_i$ of each of these patterns is calculated before a run.
Thus the number of patterns with fitness in the interval $(E, E+dE)$, denoted as $n(E)$, 
is obtained as:
\begin{displaymath}
n(E) = \sum^{\wp}_{i=1}{d_i},
\end{displaymath}
where $d_i=1$ if $E< E_i < E+dE$ and $d_i=0$ otherwise.
In this paper, $\wp$ is set to 1000, and 
$E$ is discretized into 20 consecutive classes
plus one class in which only the fitness 1 is assigned.
See Figure \ref{histogram} (A) in the next section as an example.
The addition of the class for the perfect fitness value is a novelty introduced in this paper.

Then, we define the logarithmic scaled frequency of an arbitrary configuration $x$ as
\begin{displaymath}
  S(x) = \ln n(E(x)),
\end{displaymath}
where $E(x)$ is a fitness level that $x$ belongs to.
We call this an entropy of the configuration, following Lee et al.~\cite{lee}.
Or 
\begin{displaymath}
  S(E) = \ln n(E)
\end{displaymath}
gives entropy of each fitness level.

Search is started by sampling a test configuration $x_0$ at random, 
and the entropy $S(x_0)$ is calculated.

In each generation, two parent patterns are repeatedly chosen at random, 
and these two parents produce an offspring patterns $x$ 
by being recombined with uniform crossover \cite{sysw}.
That is, two parents, $(u_1, \cdots, u_n)$ and $(v_1, \cdots, v_n)$, 
produce an offspring $(w_1, \cdots, w_n)$ so that $w_i$ is either $u_i$ or $v_i$ with equal probability.
The offspring is occasionally mutated by flipping a bit chosen with probability $p_m$, yielding $x^{\prime}$.

The selection of these offspring is made as follows.
If $S(x^{\prime}) < S(x_0)$ then $x^{\prime}$ is accepted; else if 
\begin{displaymath}
r = \frac {\exp (S(x_0))}  {\exp (S(x^{\prime}))} < 1
\end{displaymath}  
then $x^{\prime}$ is accepted with the probability $r$;
otherwise $x^{\prime}$ is rejected.

When $x^{\prime}$ is accepted,  $n(E(x^{\prime}))$ is incremented and $x_0$ is replaced with $x^{\prime}$,
unless $x^{\prime}$ is one of the global peaks.
It is important to note that $n(E)$ for the last class assigned for the fitness 1 always remains 0.
In this way, the more the offspring selected from a level, 
the smaller the chance to be selected from the level,
except for the level of the perfect fitness.

The process of producing new $x^{\prime}$ is repeated until 
the number of accepted offspring reaches the number of population 
and all these offspring construct the next generation.
Then the cycle of reconstructing a new population 
and restarting the search is iterated.
In the meantime, entropy $S(E)$ is updated every $\cal G$ generations such that
\begin{displaymath}
S(E)= \left\{
\begin{array}{ll}
         S(E)          & \qquad \mbox{if} \hspace{.05in} n(E)=0, \\
         S(E)+\ln n(E) & \qquad \mbox{otherwise}.
\end{array} \right.
\end{displaymath}
Generation is iterated until  the global optimum is found or a set maximum number of generation has been reached.

\section{Results and Discussion}
\noindent
Experiments were carried out on networks with 49 neurons.
For a given set of $p$ patterns, a configuration of synaptic weights in a network is calculated by the Hebb's rule.
Thus, the network with 49 neurons now stores these $p$ patterns as fixed point attractors,
so far as $p$ doses not exceed the storage capacity. 
Namely, the fitness landscape defined on pattern space has $p$ global peaks exactly at the location of the patterns.
Note here that if we employ the Hebb's rule, the network of this size can store a maximum of eight patterns.

The parameters of the GA employed in this paper are as follows.
The population size is 200; mutation probability $p_m$ is set to 0.05;
entropy $S(E)$ is updated every ${\cal G} = 10$ generations;
and the number of iterations is limited to a maximum of 12,000 generations.

For a specific number of patterns $p$, starting with $p=6$, 
we studied 10 runs with varying random number seed at the start of each run.
The experiments are iterated with $p$ being incremented.
As results, we observed that all the 10 runs for each $p$ up to $p=8$ succeeded in 
locating one of the embedded peaks.

\subsection{Difficulty of the Search}
\noindent
The distribution of the number of individuals in each fitness classes $n(E)$ shown in Figure \ref{histogram} (A)
in the subsection \ref{third} below
gives us the bird's-eye view of our fitness landscape.
The histogram says that the fitness values were distributed among fitness level 3 to 12,
which implies fitness distribution ranges from 0.1 to 0.6.
Peaks whose hight is 1 are too narrow to be viewed.

The difficulty to search for the peaks can be evaluated by a simple hill-climbing walk.
Davis \cite{davi} pointed out that some problems can be solved more easily 
with a simple hill-climber than a genetic algorithm.
So, we experimented a random hill climbing on the landscape.
Starting at a randomly chosen position in the space,
a point is mutated 200 times, and then moves to the highest fitness point among mutants.
We repeated a run varying $p$, but all we observed were early stagnations to a local optimum. 
The solutions of our problem were not reachable by a simple hill-climber,
which satisfies Whitley et al.'s \cite{whit} demand that test functions should be resistant to simple hill-climbing.
In Figure \ref{hillclimb}, we show an example of the hill-climbing search.
\begin{figure}[hbtp]
 \begin{center}
   \epsfile{file=5100-rmhc.eps,width=8.0cm,height=10.0cm}
   \caption{Fitness during a  hill climbing. Exploration of fitness landscape that has 8 peaks.}
   \label{hillclimb}
 \end{center}
\end{figure}

\subsection{Fitness Evolution}
\noindent
In Figure \ref{evolution}, we show a typical example of the evolution under entropic sampling.
The maximum fitness and average fitness are plotted against generation for $p=8$.
At generation 29, the maximum fitness reaches the value 1.
We can see that average fitness also reaches the value 1 at generation 43, which
implies that all the population converge to fitness 1.
Then we checked whether these peaks are identical or not.
We had expected that the algorithm would locate multiple peaks for some reasons, 
but so far all we have observed is the convergence to only one of the peaks.

\begin{figure}[hbtp]
 \begin{center}
   \epsfile{file=fitness.eps,width=8.0cm,height=10.0cm}
   \caption{The best and average fitness in the population plotted against generation during 
            an evolution  under the entropic sampling. Exploration of fitness landscape that has 8 peaks.}
   \label{evolution}
 \end{center}
\end{figure}

For comparison purposes, 
we also show an example of evolution under a simple genetic algorithm in Figure \ref{evolution-sga} 
in which the truncation selection \cite{MSV} is employed.
The truncation selection is a selection scheme that we have employed in our exploration
of weight space as a kind of elitist strategy
where two parents are chosen {\it randomly} from the best $T$\% of the population
(we usually set $T$ to 40),
and mated randomly (except for itself) to produce one offspring. 

We can see that the task to locate one of the global peaks is rather easy.
From 20 to 30 iterations were enough to find one of the peaks whichever selection scheme may be used.
The difference is that if we use the truncation selection, 157 out of 200 individuals locate a peak 
when a run terminated at 12,000 generation,
while all 200 individuals converge to one peak at generation 43 if we use the entropic sampling scheme. 

\begin{figure}[hbtp]
 \begin{center}
   \epsfile{file=fit-simple.eps,width=8.0cm,height=10.0cm}
   \caption{The best and average fitness in the population plotted against generation during 
            an evolution  under the truncation selection. Exploration of fitness landscape that has 8 peaks.}
   \label{evolution-sga}
 \end{center}
\end{figure}

\subsection{Fitness Distribution} \label{third}
\begin{figure}[hbtp]
 \begin{center}
   \epsfile{file=histogram-0.eps,width=8.0cm,height=4.6cm}
   \epsfile{file=histogram-1.eps,width=8.0cm,height=4.6cm}
   \epsfile{file=histogram-2.eps,width=8.0cm,height=4.6cm}
   \epsfile{file=histogram-3.eps,width=8.0cm,height=4.6cm}
   \epsfile{file=histogram-4.eps,width=8.0cm,height=4.6cm}
   \caption{Fitness distribution as snapshots during an evolution.}
   \label{histogram}
 \end{center}
\end{figure}

\noindent
We now look at how an evolution proceeds under entropic sampling.
As mentioned earlier, if many offspring are selected from a sub-population of a certain specific fitness level, 
then the entropy of the sub-population decreases and hence 
the probability of selecting offspring from the sub-population becomes small.
This allows us to increase fitness diversity in the population.

In Figure \ref{histogram}, we show  distributions of the number of individuals
over 20 fitness levels as snapshots during an evolution.
Here, we divided fitness range from 0 to 1 into 21 levels,
so that the fitness values in the interval $[0, 0.05)$ belong to the fitness level 0,
the fitness values in the interval $[0.05, 0.1)$ belong to the fitness level 1, and so on.
Note that the fitness level 21 is assigned only for the fitness value 1.
The distribution before an evolution are shown in Figure \ref{histogram} (A)
where individuals are distributed from fitness level 3 to 12 almost in Gaussian distribution.
The Figure \ref{histogram} (B), (C), (D), and (E) are a snapshot at generation 10, 20, 30, and 40, respectively.
We can see that the fitness diversity increases as the evolution proceeds.

As Lee et al.\ \cite{lee} wrote, one of the most important features of the entropic sampling is 
the capability to escape from local optima.
If we take a look at Figure \ref{histogram} (E), which is a snapshot at an almost final stage of the evolution,
we notice the fitness level 19 dominates the distribution.
We surmise many local peaks in our landscape lie in this fitness level.
And accumulated number of samples in the level prevents the algorithm from being stagnated to these local optima,
which is made by mutation operation in usual selection schemes based on fitness.
The fitness level 20, on the other hand, remains empty throughout the evolution,
which keeps giving the algorithm an evolution pressure to sample the global optimum.

\section{Conclusion}
\noindent
We have described an application of a GA that employs entropic sampling
to the Hopfield model of associative memory.
As a result, we have observed the entropic sampling worked effectively for the GA
to locate one of the peaks embedded in the fitness landscape.

Although we have explored pattern space as a test case in this paper,
what we are concerning is to obtain multiple solutions of weight configuration in weight space,
and the approach we have taken in this paper
can be extended in the problem of searching for the solutions in weight space.

Since it is not according to the fitness but the entropy that the entropic sampling selects individuals, 
low fitness individuals can have a high possibility to survive.
Hence, the GA using entropic sampling can keep the diversity in a population.
We suppose this will enable us to locate multiple solutions when we explore fitness landscape defined on weight space.
In the experiment in this paper, however, all the individuals in the population converged on one of the embedded peaks.
This is probably because we did not update the entropy of the fitness class to which the global peaks belong.
More improvements of the algorithm are still awaiting exploration.

We believe that this study contributes to the new analysis of the associative memory model.
In fact, many issues still remain open, despite the extensive discussions since the proposal of the model by Hopfield.
If we locate, for example, all the solutions of synaptic weights which give a network a function of associative memory, 
it will give an insight into the number and distribution of the solutions in synaptic weight space, 
about which we have now only limited information.

\acknowledgements{}
\noindent
We thank Peter Davis at Advanced Telecommunication Research Institute (ATR)
for providing us great insights into the dynamics of the Hopfield neural networks.

\begin{thebibliography}{00}
\bibitem{koml}J.~Koml\'{o}s, and R. Paturi (1988)
{\em Convergence Results in an Associative Memory Model.}
Neural Networks, 1, pp.239--250.

\bibitem{hopf}J. J. Hopfield (1982) 
{\em Neural Networks and Physical Systems with Emergent Collective Computational Abilities.}
Proceedings of the National Academy of Sciences, USA, 79,  pp.2554-2558.

\bibitem{hebb}D.~O.~Hebb (1949)
{\em The Organization of Behavior. Wiley.}

\bibitem{lee}C.-Y. Lee, and  S.-K. Han (1997)
{\em Genetic-Entropic Analysis: An Application to NK-model and the Statistical Analysis.}
Proceedings of the 7th International Conference on Genetic Algorithms, pp.254--259.

\bibitem{kauf} S. A. Kauffman, and S. Levin (1987)
{\em Towards a General Theory of Adaptive Walks on Rugged Landscapes.}
Journal of Theoretical Biology, 128, pp.11--45.

\bibitem{coop}L. N. Cooper (1973)
{\em A Possible Organization of Animal Memory and Learning.}
Proceedings of the 24th Nobel Symposium on Collective Properties of Physical Systems,
pp.252--264.

\bibitem{koho} T. Kohonen, and M. Ruohonen (1973)
{\em Representation of Associated Data by Matrix Operators.}
IEEE Trans. Computers, C-22(7),  pp.701--702.

\bibitem{pers}L. Personnaz, I. Guyon, and G. Dreyfus (1985)
{\em Information Storage and Retrieval in Spin-Glass like Neural Networks.}
Journal of Physique Lettre, 
46, pp.L359-L365.

\bibitem{wrig}S. Wright (1932)
{\em The Roles of Mutation, Inbreeding, Crossbreeding and Selection in Evolution.}
Proceedings of the 6th International Congress of Genetics, pp.356--366.

\bibitem{mack}C. Macken,  P. Hagan, and A. Perelson (1991)
{\em Evolutionary Walks on Rugged Landscapes.}
SIAM Journal of Applied Mathematics, 51, pp.799--827.

\bibitem{font}W. Fontana, and P. Schuster (1987)
{\em A Computer Model of Evolutionary Optimization.}
Biophysical Chemistry, 26, pp.123--147.

\bibitem{mayn}J. Maynard Smith (1970)
{\em Natural Selection and the Concept of a Protein Space.}
Nature, 225, pp.563--564.

\bibitem{bray}A. Bray, and M. Moore (1980)
{\em Metastable States in Spin Glasses.}
Journal of Physics C: Solid State Physics 13, pp.L469--L476.

\bibitem{wein}E. D. Weinberger (1990)
{\em Correlated and Uncorrelated Fitness Landscapes and How to Tell the Difference.}
Biological Cybernetics, 63, pp.325--336.

\bibitem{matt}D. C. Mattis (1976)
{\em Solvable Spin Systems with Random Interactions.}
Physics Letters, 56A, pp.421--422.

\bibitem{SK}D. Sherrington,  and S.Kirkpatrick (1975) 
{\em Solvable Model of a Spin Glass.}
Physical Review Letters, 35, pp.1792--1796.

\bibitem{amit}D. J. Amit (1989)
{\em Modeling Brain Function: The World of Attractor Neural Networks.}
Cambridge University Press.

\bibitem{gard}E. Gardner (1988)
{\em The Phase Space of Interactions in Neural Network Models.}
Journal of Physics, 21A, pp.257--270.

\bibitem{imad-1}A. Imada, and K. Araki (1995)
{\em Genetic Algorithm Enlarges the Capacity of Associative Memory.}
Proceedings of the 6th International Conference on Genetic Algorithms, pp.413--420.

\bibitem{imad-12}A. Imada, and K. Araki (1997)
{\em Evolution of Hopfield Model of Associative Memory by the Breeder Genetic Algorithm.}
Proceedings of the 7th International Conference on Genetic Algorithms, pp.784--791.

\bibitem{imad-5}A. Imada, and K. Araki (1996)
{\em Basin of Attraction of Associative Memory as it Evolves from Random Weights.}
Proceedings of the 1st Asia-Pacific Conference on Simulated Evolution and Learning,
pp.271-278.

\bibitem{sysw}G. Syswerda (1989)
{\em Uniform Crossover in Genetic Algorithms.}
Proceedings of the 3rd International Conference on Genetic Algorithms, pp.2--9.

\bibitem{MSV}H. M\"{u}hlenbein, and D. Schlierkamp-Voosen (1996)
{\em Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization.}
Evolutionary Computation, 1,  pp.25--49.

\bibitem{davi}L. Davis (1991)
{\em Bit-Climbing, Representation Bias, and Test Suite Design.}
Proceedings of the 4th International Conference on Genetic Algorithms,
pp.18--23.

\bibitem{whit}D. Whitley,  K. Mathias, S. Rana, and J. Dzubera (1995)
{\em Building Better Test Functions.}
Proceedings of the 6th International Conference on Genetic Algorithms,
pp.239--246.

\end{thebibliography}

\end{document}
