Abstract by Chris Anderson
PCA-inspired Optimization of Neural Networks
The basic elements of a neural network are neurons, which are connected with weights, also known as parameters; a cost function to measure how much the network has "learned"; and an optimizer, which adjusts the weights to improve the score given by the cost function. Optimization of a neural network takes time, sometimes on the order of days. We present work on a new optimization technique inspired by Principal Component Analysis, with hopes to minimize the time it takes to train neural networks.