Deep neural networks as kernel machines: An approach to explain the mysteriously superb performance of ANNs

Ms. Gitte Kremling
Graduate Teaching Assistant
University of Wisconsin – Milwaukee

“Due to their extremely good performance, artificial neural networks (ANNs) have been a hot topic in the field of machine learning for several years now. They are used in a lot of different applications such as image classification, speech recognition and translating languages. However, we are still missing a thorough understanding of the theory behind their success. It is particularly mysterious that they perform so well despite their high number of parameters. According to standard machine learning theory, algorithms involving a high number of parameters are at risk of overadapting to the training data and thus not being able to generalize well when given new data; a phenomenon known as overfitting. In this talk, we will discuss an approach to explain why ANNs do not seem to suffer from this issue. It relies on a theorem showing that neural networks converge to kernel machines, a known tool in machine learning, as we use a network with infinite width and depth.”