FI Computational Methods and Data Science Journal Club: Larry Saul (CCM)

Tuesday Dec 13, 2022, 3:00 PM → 5:00 PM America/New_York

5th Floor Classroom/5-Flatiron Institute (162 5th Avenue)

Rescheduled from November 8th

FI Computational Methods and Data Science Journal Club

Flatiron Institute, 162 5th Avenue

Speaker: Larry Saul (CCM)

Title: A geometrical connection between sparse and low-rank matrices and its uses for machine learning

Abstract: Many problems in high dimensional data analysis can be formulated as a search for structure in large matrices. One important type of structure is sparsity; for example, when a matrix is sparse, with a large number of zero elements, it can be stored in a highly compressed format. Another type of structure is linear dependence; when a matrix is low-rank, it can be expressed as the product of two smaller matrices. It is well known that neither one of these structures implies the other. But can one find more subtle connections by looking beyond the canonical decompositions of linear algebra?

In this talk, I will consider when a sparse nonnegative matrix can be recovered from a real-valued matrix of significantly lower rank. Of particular interest is the setting where the positive elements of the sparse matrix encode the similarities of nearby points on a low dimensional manifold. The recovery can then be posed as a problem in manifold learning—namely, how to learn a similarity-preserving mapping of high dimensional inputs into a lower dimensional space. I will describe an algorithm for this problem based on a generalized low-rank decomposition of sparse matrices. This decomposition has the interesting property that it can be encoded by a neural network with one layer of rectified linear units; since the algorithm discovers this encoding, it can also be viewed as a layerwise primitive for deep learning. Finally, I will apply the algorithm to data sets where vector magnitudes and small cosine distances have interpretable meanings (e.g., the brightness of an image, the similarity to other words). On these data sets, the algorithm is able to discover much lower dimensional representations that preserve these meanings.

Bio: Lawrence Saul is a Senior Research Scientist in the Center for Computational Mathematics (CCM) at the Flatiron Institute. He joined CCM in July 2022 as a group leader in machine learning; previously, he was a Professor and Vice Chair in the Department of Computer Science and Engineering at UC San Diego.
 

Attendee Instructions:
FI employees are welcome to attend in person. Please email ccaadmin@flatironinstitute.org for the Zoom link if you wish to attend remotely.
Visitors (w/out an FI badge) please email ccaadmin@flatironinstitute.org 24hrs in advance to be registered to the building or to obtain Zoom information.

Additional Information:
COVID Policy: By making entry to our buildings all staff, vendors and guests will implicitly attest to being symptom/COVID free. Vaccination status will no longer be validated as a condition of entry. However, all staff and affiliates are strongly encouraged to remain up to date with their vaccination boosters, according to their individual eligibility.

Age Restriction: All employees, visitors, event attendees and vendors are required to be above the age of eighteen for entry into our building(s). Photo ID with birthdate will be required by security upon arrival to our building. For nursing mothers, please reach out to an admin to arrange an exception.