#### IMSc Webinar

#### Recent Developments in the Mathematics of Neural Nets

#### Anirbit Mukherjee

##### University of Pennsylvania

*zoom.us/j/91782375389?pwd=aUo4UWZKZjh1SlBYWkV2QlBnY3VyZz09*

Meeting ID: 917 8237 5389; password: Turing

A profound mathematical mystery of our times is to be able to

explain the phenomenon of training neural nets i.e ``deep-learning”.

The dramatic progress of this approach in the last decade has gotten

us the closest we have ever been to achieving "artificial

intelligence". But trying to reason about these successes - for even

the simplest of nets - immediately lands us into a plethora of

extremely challenging mathematical questions, typically about

discrete stochastic processes. In this talk we will describe the

various themes of our work in provable deep-learning.

We will start with a brief introduction to neural nets and then see

glimpses of our initial work on understanding neural functions, loss

functions for autoencoders and algorithms for exact neural training.

Next, we will explain our recent result about how under mild

distributional conditions we can construct an iterative algorithm

which can be guaranteed to train a ReLU gate in the realizable

setting in linear time while also keeping track of mini-batching -

and its provable graceful degradation of performance under a

data-poisoning attack. We will show via experiments the intriguing

property that our algorithm very closely mimics the behaviour of

Stochastic Gradient Descent (S.G.D.), for which similar convergence

guarantees are still unknown.

Lastly, we will review this very new concept of "local elasticity"

of a learning process and demonstrate how it appears to reveal

certain universal phase transitions during neural training. Then we

will introduce a mathematical model which reproduces some of these

key properties in a semi-analytic way. We will end by delineating

various exciting future research programs in this theme of

macroscopic phenomenology with neural nets.

Done