Representational Power of Feedforward Networks

• Boolean functions: 2 layers of units, number of hidden nodes grows exponentially in the number of inputs
• Continuous Functions: every bounded continuous function to arbitrary error* in two layers of units
• Arbitrary Functions: arbitrary accuracy* by a network with 3 layers of units - based on linear combination of many localized functions
• But the network weight variables reachable from the initial weight values may not include all possible weight vectors