BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CERN//INDICO//EN
BEGIN:VEVENT
SUMMARY:Rishi Sonthalia Talk
DTSTART:20230331T140000Z
DTEND:20230331T160000Z
DTSTAMP:20241103T215000Z
UID:indico-event-3632@indico.flatironinstitute.org
DESCRIPTION:Speakers: Rishi Sonthalia (UCLA)\n\n Bio: Rishi Sonthalia is
currently a Hedrick Assistant Adjunct Professor at UCLA under Andrea Berto
zzi\, Jacob Foster\, and Guido Montufar. He obtained by Ph.D. in Applied a
nd Interdisciplinary Mathematics from the University of Michigan. His advi
sors were Anna C. Gilbert and Raj Rao Nadakuditi. He did his undergraduate
degree at Carnegie Mellon University where he obtained a B.S. in Discrete
Math and Computer Science.Title: Surprises in Structured denoising and Do
uble Descent for Linear ModelsAbstract: In this talk\, we look at the prob
lem of structured denoising for low rank data using linear models. First\,
when training an unregularized denoising feedforward neural network\, we
empirically show that the generalization error versus number of training d
ata points is a double descent curve. We formalize the question of how man
y training data points and the amount of noise should be used by looking a
t the generalization error for denoising noisy test data. Prior work on co
mputing the generalization error focus on adding noise to target outputs.
However\, adding noise to the input is more in line with current pre-train
ing practices. In the linear (in the inputs) regime\, we provide an asympt
otically exact formula for the generalization error for rank r data.From t
his\, we derive a formula for the amount of noise that needs to be added t
o the training data to minimize the denoising error. This results in the e
mergence of a shrinkage phenomena for improving the performance of denoisi
ng DNNs by making the training SNR smaller than test SNR. Further\, we see
that the amount of shrinkage (ratio of train to test SNR) follows a doubl
e descent curve as well.\n\nhttps://indico.flatironinstitute.org/event/363
2/
LOCATION:3rd Floor Classroom/3-Flatiron Institute (162 5th Avenue)
URL:https://indico.flatironinstitute.org/event/3632/
END:VEVENT
END:VCALENDAR