Evolving, Training and Designing Neural Network Ensembles

Xin Yao

CERCIA and Natural Computation Group School of Computer Science University of Birmingham Edgbaston, Birmingham B15 2TT, UK (http://www.cs.bham.ac.uk/ xin)

Abstract

Previous work on evolving neural networks has focused on single neural networks. However, monolithic neural networks are too complex to train and evolve for large and complex problems. It is often better to design a collection of simpler neural networks that work cooperatively to solve a large and complex problem. The key issue here is how to design such a collection automatically so that it has the best generalisation. This talk introduces work on evolving neural network ensembles, negative correlation learning, and multi-objective approaches to ensemble learning. The links among different learning algorithms are discussed. Online/incremental learning using ensembles will also be presented briefly.

Selected References:

On negative correlation: Y. Liu and X. Yao, “Simultaneous training of negatively correlated neural networks in an ensemble,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 29(6):716-725, December 1999.

On negative correlation and evolution: Y. Liu, X. Yao and T. Higuchi, “Evolutionary Ensembles with Negative Correlation Learning,” IEEE Transactions on Evolutionary Computation, 4(4):380-387, November 2000.

On constructive ensemble learning: Md. Monirul Islam, X. Yao and K. Murase, “A constructive algorithm for training cooperative neural network ensembles,” IEEE Transactions on Neural Networks, 14(4):820-834, July 2003.

On multi-objective approaches to ensemble learning: A Chandra and X. Yao, “Ensemble learning using multi-objective evolutionary algorithms,” Journal of Mathematical Modelling and Algorithms, 5(4):417-445, December 2006.

On incremental learning using ensembles: F. L. Minku, H. Inoue and X. Yao, “Negative correlation in incremental learning,” Natural Computing, 8(2):289-320, June 2009.

On ensemble pruning: H. Chen, P. Tino and X. Yao, “Predictive Ensemble Pruning by Expectation Propagation,” IEEE Transactions on Knowledge and Data Engineering, 21(7):999-1013, July 2009.

On evolving ensembles: X. Yao and Md. M. Islam, “Evolving artificial neural network ensembles,” IEEE Computational Intelligence Magazine, 3(1):31-42,February 2008.

Biosketch of the Speaker:

Xin Yao is a Professor (Chair) of Computer Science at the University of Birmingham, UK.

Currently he is the Director of CERCIA (the Centre of Excellence for Research in Computational Intelligence and Applications, http://www.cercia.ac.uk) at the University of Birmingham, UK, which is specialised in applied research and knowledge transfer. He is an IEEE Fellow and a Distinguished Lecturer of IEEE Computational Intelligence Society. He won the 2001 IEEE Donald G. Fink Prize Paper Award and several other best paper awards. He was a Cheung Kong Scholar (Changjian Chair Professor) of the Ministry of Education of China, a Distinguished Visiting Professor (Grand Master Chair Professorship) of USTC in Hefei, and a Distinguished Visiting Professor of Yuan Ze University, Taiwan. In his spare time, he did the voluntary work as the Editorin-Chief (2003-08) of IEEE Transactions on Evolutionary Computation, and is an associate editor or editorial board member of 12 international journals, and the editor of the World Scientific book series on ”Advances in Natural Computation”. He has been invited to give more than 55 keynote/plenary speeches at international conferences in many countries.

His major research interests include evolutionary computation and neural network ensembles. He has more than 300 refereed publications in journals and conferences. His work has been supported by AWM, EPSRC, EU, Royal Society, Chinese Academy of Sciences, NSFC, Honda, Marconi, BT, Thales and Severn Trent Water.