LCV Lab Meeting: Speaker Florentin Guth (CCN Summer Intern)

Wednesday Jun 8, 2022, 1:00 PM → 2:30 PM America/New_York

4th Floor Classroom/4-Simons Foundation (160 5th Avenue)

Please join us for today's LCV group meeting with our guest speaker, CCN Summer Intern, Florentin Guth who will be presenting. Title and Abstract below! Please gather in the CCN Classroom (our 4th Floor Large Conference Room A will be undergoing AV repairs today).


Title: Phase Collapse in Neural Networks


Abstract: Deep convolutional classifiers linearly separate image classes and improve accuracy as depth increases. They progressively reduce the spatial dimension whereas the number of channels grows with depth. Spatial variability is therefore transformed into variability along channels. A fundamental challenge is to understand the role of non-linearities together with convolutional filters in this transformation.
ReLUs with biases are often interpreted as thresholding operators that improve discrimination through sparsity. We demonstrate that it is a different mechanism called phase collapse which eliminates spatial variability while linearly separating classes. We show that collapsing the phases of complex wavelet coefficients is sufficient to reach the classification accuracy of ResNets of similar depths. However, replacing the phase collapses with thresholding operators that enforce sparsity considerably degrades the performance.