Evolving, Training and Designing Neural Network Ensembles

Xin Yao (http://www.cs.bham.ac.uk/~xin)
CERCIA and Natural Computation Group 
School of Computer Science
University of Birmingham
Edgbaston, Birmingham B15 2TT, UK

Abstract:

Previous work on evolving neural networks has focused on single neural
networks. However, monolithic neural networks are too complex to train
and evolve for large and complex problems. It is often better to design a
collection of simpler neural networks that work cooperatively to solve a 
large and complex problem. The key issue here is how to design such a 
collection automatically so that it has the best generalisation. 
This talk introduces work on evolving neural network ensembles, 
negative correlation learning, and multi-objective approaches to 
ensemble learning. The links among different learning algorithms are discussed.
Online/incremental learning using ensembles will also be presented briefly.

Selected References:

On negative correlation: Y. Liu and X. Yao, ``Simultaneous training of
negatively correlated neural networks in an ensemble,'' IEEE Transactions
on Systems, Man, and Cybernetics, Part B: Cybernetics, 29(6):716-725,
December 1999.

On negative correlation and evolution: Y. Liu, X. Yao and T. Higuchi,
``Evolutionary Ensembles with Negative Correlation Learning,'' IEEE
Transactions on Evolutionary Computation, 4(4):380-387, November 2000.

On constructive ensemble learning: Md. Monirul Islam, X. Yao and K.
Murase, ``A constructive algorithm for training cooperative neural network
ensembles,'' IEEE Transactions on Neural Networks, 14(4):820-834, July 2003.

On multi-objective approaches to ensemble learning: A Chandra and X. Yao,
``Ensemble learning using multi-objective evolutionary algorithms,'' Journal of
Mathematical Modelling and Algorithms, 5(4):417-445, December 2006. 

On incremental learning using ensembles: F. L. Minku, H. Inoue and X. Yao,
``Negative correlation in incremental learning,'' Natural Computing,
8(2):289-320, June 2009.

On ensemble pruning: H. Chen, P. Tino and X. Yao, ``Predictive Ensemble Pruning
by Expectation Propagation,'' IEEE Transactions on Knowledge and Data
Engineering, 21(7):999-1013, July 2009.

On evolving ensembles: X. Yao and Md. M. Islam, ``Evolving artificial neural
network ensembles,'' IEEE Computational Intelligence Magazine,
3(1):31-42,February 2008.

Biosketch of the Speaker:

Xin Yao is a Professor (Chair) of Computer Science at the University of
Birmingham, UK. 
Currently he is the Director of CERCIA (the Centre of Excellence for Research
in Computational Intelligence and Applications, http://www.cercia.ac.uk) at the
University of Birmingham, UK, which is specialised in applied research and
knowledge transfer. He is an IEEE Fellow and a Distinguished Lecturer of IEEE
Computational Intelligence Society. He won the 2001 IEEE Donald G. Fink Prize
Paper Award and several other best paper awards. He was a Cheung Kong Scholar
(Changjian Chair Professor) of the Ministry of Education of China, a
Distinguished Visiting Professor (Grand Master Chair Professorship) of USTC
in Hefei, and a Distinguished Visiting Professor of Yuan Ze University, 
Taiwan. In his spare time, he did the voluntary work as the Editor-in-Chief 
(2003-08) of IEEE Transactions on Evolutionary Computation, and is an 
associate editor or editorial board member of 12 international journals, 
and the editor of the World Scientific book series on "Advances in Natural 
Computation". He has been invited to give more than 55 keynote/plenary 
speeches at international conferences in many countries.

His major research interests include evolutionary computation and neural network
ensembles. He has more than 300 refereed publications in journals and 
conferences. His work has been supported by AWM, EPSRC, EU, Royal Society, 
Chinese Academy of Sciences, NSFC, Honda, Marconi, BT, Thales and Severn Trent 
Water.