BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CERN//INDICO//EN
BEGIN:VEVENT
SUMMARY:CCN Seminar: Francesca Mignacco presents ""Statistical physics ins
ights on stochastic gradient descent""
DTSTART;VALUE=DATE-TIME:20211210T173000Z
DTEND;VALUE=DATE-TIME:20211210T190000Z
DTSTAMP;VALUE=DATE-TIME:20220121T074200Z
UID:indico-event-2743@indico.flatironinstitute.org
DESCRIPTION:Dear CCN Colleagues\,\n\nPlease welcome guest speaker and cand
idate for a Flatiron Research Fellow position here at Flatiron\, Francesca
Mignacco\, Institute of Theoretical Physics\, Paris-Saclay University\,
for our CCN Seminar. She will be presenting\, "Statistical physics insight
s on stochastic gradient descent." Abstract below! \n\n\nAbstract: Artifi
cial neural networks (ANNs) trained with gradient-based algorithms have ac
hieved impressive performances in a variety of applications. In particular
\, the stochastic gradient-descent (SGD) algorithm has proved to be surpri
singly efficient in navigating high-dimensional loss landscapes. However\,
the theory behind this practical success remains largely unexplained. A g
eneral consensus has arisen that the answer requires a detailed descriptio
n of the trajectory traversed during training. This task is highly nontriv
ial for at least two reasons. First\, the high dimension of the parameter
space where ANNs operate defies standard mathematical techniques. Second\,
SGD navigates a non-convex loss landscape following an out-of-equilibrium
dynamics with a complicated state-dependent noise. In this talk\, I will
consider prototypical learning problems that are amenable to an exact char
acterisation. I will show how methods from spin glass theory can be used t
o derive a low-dimensional description of the network performance and the
learning dynamics of gradient-based algorithms\, including multi-pass SGD.
I will discuss how different sources of algorithmic noise affect the perf
ormance of the network in a benchmark high-dimensional non-convex task (si
gn retrieval) and how to characterise SGD noise via an effective fluctuati
on-dissipation relation holding at stationarity.\n\nDue to limited space\,
please feel welcome to tune in remotely. Zoom credentials in the calendar
invite. \nhttps://indico.flatironinstitute.org/event/2743/
URL:https://indico.flatironinstitute.org/event/2743/
END:VEVENT
END:VCALENDAR