A Primer on the Mathematics of Artificial Neural Networks

Daniel Quigley
PhD Student
University of Wisconsin-Milwaukee

Artificial neural networks (ANNs, or, simply, neural networks) are ubiquitous, not least of all in the context of modern machine learning. This presentation is a primer on the mathematics that underlie the mechanics of relatively simple feedforward ANNs. A sketch of the proof for the universal approximation theorem is given, which states that a (fully connected) ANN with at least one hidden layer (of a sufficient number of neurons), together with a non-linear activation function, can approximate any continuous function on a compact set to arbitrary accuracy. This presentation contributes to the movement for providing mathematical explanations and descriptions of ANNs, favoring a functional analytical and well-founded framework at the expense of algorithmic aspects of deep learning otherwise concerned with identifying the most suitable deep ANNs for specific applications.