Fire Together, Wire Together

Miikkulainen and Sirosh's essential insight has been to adapt a learning mechanism first proposed in 1949, called Hebbian learning, to self-organization in the visual cortex. The Hebbian mechanism adjusts connections according to the idea that neurons active at the same time increase the strength of their connection. "You can think of it as ﾔNeurons that fire together, wire together,'" says Sirosh. "As a result of this process, connections that are required are retained and others are eliminated."

They implemented their Hebbian algorithm in a network model they call LISSOM (Laterally Interconnected Synergetically Self-Organizing Map). LISSOM models a sheet of neurons in the primary visual cortex -- typically 65,000 neurons with 700 million interconnections. Each neuron connects to "receptors" in two fields, representing the right and left retina, and to other neurons in the network. LISSOM is the first model of the visual cortex to simulate self-organization of these neuron-to-neuron or "lateral" connections.

To date, LISSOM -- which performs at a sustained speed of eight billion calculations a second on 256 T3D processors -- has shown a promising ability to reproduce observed patterns of neural organization. What is most interesting is that a single learning mechanism can reproduce various features of self-organization.

"The model allows us to predict how lateral connections organize in the visual cortex," says Miikkulainen, "and what visual knowledge is stored in them." In the brain, this information is preliminary to processes that organize distinct visual features into coherent wholes. LISSOM's success at modeling self-organization suggests that neural modeling will play a vital role as vision research presses on to explore these higher level processes.

Self-Organization of the Orientation Map
Using oriented light spots as input, LISSOM modeled how orientation preference and neuron-to-neuron connection patterns develop in the cortex. The color of each neuron in the network (192 x 192 neurons), from red to magenta to blue to green, corresponds to its orientation preference from zero to 180 degrees. The small white dots show lateral connections of the neuron marked with a big white dot.

Initially (left) the orientation preferences were random and lateral connections covered almost the entire map. After a series of several thousand inputs, the neurons organize (right) into orientation columns, and lateral connections link areas of similar orientation preference. These patterns agree in key features with maps obtained by experimental imaging.

go back to the main page