The Cascade-Correlation Learning Architecture

Scott E. Fahlman and Christian Lebiere

Reviewer

John Trevithick (j.trevithick@auckland.ac.nz)

Reference

Fahlman, S. E. and Lebiere, C. (1990) "The Cascade-Correlation Learning Architecture", in Advances in Neural Information Processing Systems 2, D. S. Touretzky (ed.), Morgan Kaufmann Publishers, Los Altos CA, pp.524-532.
Downloaded from http://www.cs.cmu.edu/~sef/sefPubs.htm

Keywords

Cascade-Correlation, ANNs, feature-detector, constructive algorithm, topology, back-propagation.

Related Papers

Fahlman, S. E. (1991) "The Recurrent Cascade-Correlation Architecture" in Advances in Neural Information Processing Systems 3, D. S. Touretzky (ed.), Morgan Kaufmann, Los Altos CA, pp. 190-196.
Fahlman, S. E. (1988) "Faster-Learning Variations on Back-Propagation: An Empirical Study" in Proceedings, 1988 Connectionist Models Summer School, Morgan-Kaufmann, Los Altos CA.
S. Baluja and S.E. Fahlman, “Reducing network depth in the cascade-correlation”. Technical Report CMU-CS-94-209, Carnegie Mellon University (1994).

Summary

Why is Back-Propagation Learning So Slow? The authors propose some key causes behind the need for BPL's many epochs.
Description of Cascade-Correlation. Describes the algorithm with reference to cascade architecture and correlation of residual error signal.
Benchmark Results. Shows outcome of testing against two known problems: the two-spirals problem and the N-input parity problem.
Discussion. Briefly summarises perceived advantages of the algorithm.
Relation To Other Work. The authors identify the origins of many of their base ideas and approaches.

Abstract:
Cascade-Correlation is a new architecture and supervised learning algorithm for artificial neural networks. Instead of just adjusting the weights in a network of fixed topology, Cascade-Correlation begins with a minimal network, then automatically trains and adds new hidden units one by one, creating a multi-layer structure. Once a new hidden unit has been added to the network, its input-side weights are frozen. This unit then becomes a permanent feature-detector in the network, available for producing outputs or for creating other, more complex feature detectors. The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the connections of the network.

Evaluation

Strengths of the paper:

Both the paper's format and the argument's form are clearly laid out.
There is a compelling logic to the design of the algorithm.
Authors generally avoided unnecessary technical jargon or detail.

Weaknesses of the paper:

Authors didn't quantify impact of the moving-target problem.
Mention of Quickprop as feature of CasCor was unnecessary, and actually weakened their argument.
No apparent negatives of CasCor were mentioned.