##### Department of Mathematics,

University of California San Diego

****************************

### Math 278B - Mathematics of Information, Data, and Signals

## Liam Madden

#### University of British Columbia

## Memory capacity of two-layer neural networks with analytic activations

##### Abstract:

The memory capacity of a statistical model is the largest size of generic data that the model can memorize and has important implications for both training and generalization. In this talk, we will prove a tight memory capacity result for two-layer neural networks with polynomial or real analytic activations. In order to do so, we will use tools from linear algebra, combinatorics, differential topology, and the theory of real analytic functions of several variables. In particular, we will show how to get memorization if the model is a local submersion and we will show that the Jacobian has generically full rank. The perspective that is developed also opens up a path towards deeper architectures, alternative models, and training.

Host: Alex Cloninger

### October 19, 2023

### 11:30 AM

APM 2402

****************************