Neural Network Learning

• Artificial Neural Networks
• Human Brain
• Neural Network Representations
• ALVINN Neural Network
• Backpropagation Network Representations
• Appropriate Problems for ANN
• Perceptrons
• A Perceptron
• Linearly Separable
• Representational Power of Perceptrons
• Perceptron Learning Algorithms
• Perceptron Training Rule
• Intuition for Perceptron Training Rule
• Gradient Descent Algorithm
• Weight Update Rule
• Training Error
• Gradient-Descent Algorithm
• Problems with Gradient Descent
• Stochastic Gradient Descent
• Differences between GD and SGD
• Delta Rule vs. Perceptron Training Rule
• Multilayer Networks & Nonlinear Surfaces
• Multilayer Networks
• Sigmoid Threshold Unit
• Properties of the Backpropagation Algo.
• Backpropagation Algorithm
• Backpropagation Weight Training Rule
• Termination Conditions for Backprop
• Momentum
• Arbitrary Acyclic Networks
• Convergence and Local Minima
• Heuristics to Overcome Local Minima
• Representational Power of Feedforward Networks
• Hypothesis Space & Inductive Bias
• Hidden Layer Representations
• Backprop in Action
• Overfitting and Stopping Criteria
• Error Plots
• Face Recognition Task
• Input Encoding
• Output Encoding
• Network Graph Structure
• Other Algorithm Parameters
• Learned Hidden Representations
• Advanced Topics
• Summary