Network Graph Structure

• How many units and how to interconnect?
• most common - layer units with feedforward connections form every unit in one layer to every unit in the next
• the more layers the longer the training time - we choose 1 hidden layer and 1 output layer
• how many hidden units? - 3 units = 90% accuracy, 5 minutes; 30 units =  92% accuracy, 1 hr
• in general there is a minimum number of hidden units needed and above that the extra hidden units do not dramatically effect the accuracy, provided cross validation is used to determine how many gradient descent iterations should be performed, otherwise increasing the number of hidden units often increases the tendency to overfit the training data