CCM Mathematics of Deep Learning Seminar with Stéphane Mallat

America/New_York
Description

Harmonic Analysis View of Deep Network Concentration

Stephane Mallat
CCM Flatiron Institute, College de France, Ecole Normale Superieure

What mathematics can explain how deep convolutional network avoid the curse of dimensionality ? This talk will emphasize the impact of convolutional structures, leading to low-dimensional harmonic analysis. Numerical experiments indicate that deep networks concentrate data distributions. Beginning from two layer networks, we will analyze the role of known group transformations and multiscale sparse representations to explain this concentration. To simplify the mathematics, I will introduce highly structured multiscale networks, where spatial filters are not learned, but still achieve ResNet performances. Concentration can then be related to invariances and renormalization group mechanisms, which opens many questions that I will discuss.

This talk is based on a joint work with Florentin Guth and John Zarka: https://arxiv.org/abs/2012.10424

Zoom information: Join from PC, Mac, Linux, iOS or Android: https://simonsfoundation.zoom.us/j/93986481534?pwd=TVB2a2JobE5BWGdySDNyZ1JnMmZmQT09
    Passcode: 389303

The agenda of this meeting is empty