Fall 2025

Aug 29: Simone Bombari (Institute of Science and Technology Austria)

Privacy for Free in the Overparameterized Regime

Differentially private gradient descent (DP-GD) is a popular algorithm to train deep learning models with provable guarantees on the privacy of the training data. In the last decade, the problem of understanding its performance cost with respect to standard GD has received remarkable attention from the research community, which has led to upper bounds on the excess population risk in different learning settings. However, such bounds typically degrade with over-parameterization, i.e., as the number of parameters p gets larger than the number of training samples n — a regime which is ubiquitous in current deep-learning practice. As a result, the lack of theoretical insights leaves practitioners without clear guidance, leading some to reduce the effective number of trainable parameters to improve performance, while others use larger models to achieve better results through scale. In this work, we show that in the popular random features model with quadratic loss, for any sufficiently large p, privacy can be obtained for free, i.e., the with vanishing excess population risk, not only when the privacy parameter ε has constant order, but also in the strongly private setting ε = o(1). This challenges the common wisdom that over-parameterization inherently hinders performance in private learning.

September 5: Manuel Fernandez (USC)

Distance theorems and the smallest singular value of random matrices

September 12: Kabir Verchand (USC)

The dynamics of iterative algorithms with random data: Beyond first-order methods

In this talk, I will present a toolbox to analyze a broad class of iterative algorithms for high-dimensional (non)-convex optimization with random data. This class is rich enough to include general first-order methods with non-separable, Lipschitz updates (such as gradient descent and approximate message passing) as well as commonly used higher-order updates such as the proximal point method, prox-linear methods, as well as variants of alternating minimization and expectation maximization. For this class of algorithms, I will provide an exact, deterministic description of their dynamics as well as finite-sample guarantees bounding the deviation of the empirical iterates from their deterministic counterparts. Our techniques are based on a sequential variant of Gordon’s Gaussian comparison inequalities applied in conjunction with Bolthausen’s Gaussian conditioning technique. Joint work with Michael Celentano, Chen Cheng, and Ashwin Pananjady.

September 19: Mihai Cucuringu (UCLA)

Spectral methods for clustering signed/directed networks and heterogeneous group synchronization

September 26: Alexander Clay (USC)

[Undergrad Focused Talk]

Modern Approaches to Card Shuffling

We will explain some exciting connections between card shuffling and probability. Card shuffling is an old subject that combines probability with modern algebra. We will explore different shuffles through group theoretic and linear-algebraic points of view. Finally, we will investigate a model for casino card shuffling machines, and present some new research in this area.

October 3: Arthur Schichl (ETH Zürich)

Non-linear Degenerate Parabolic Flow Equations and a Finer Differential Structure on Wasserstein Spaces

We define new differential structures on the Wasserstein spaces $W_p(M)$ for $p > 2$ and a Riemannian manifold $(M,g)$. We consider a very general and possibly degenerate second-order partial differential flow equation with measure dependent coefficients to expand the notion of smooth curves and to ensure that the new differential structure is finer than the classical one. The theory allows for higher order calculus on Wasserstein spaces and admits numerical approximations in $W_p(M)$. We prove a generalized version of the Central Limit Theorem without requiring independence. We shall also present some of its economic applications.

October 10: [no classes]

October 17: Yeganeh Alimohammadi (USC)

How to Measure Differences in Rankings: A Data-Driven Extension of the Mallows Model

October 24: Alexander Tartakovsky (AGTStatConsult, Los Angeles) [Joint with CAMS Colloquium]

Optimal Sequential Tests of Multiple Composite Hypotheses for General Stochastic Models with Non-i.i.d. Observations

In many applications—such as monitoring sensor networks, detecting epidemics, or ensuring cybersecurity—it is necessary to test multiple composite hypotheses under strict error constraints. Sequential methods aim to minimize detection delay or sample size while controlling false identification risks. This talk presents a general framework for sequential multi-hypothesis testing in models with dependent and non-identically distributed observations. The resulting procedures achieve near-optimal performance in regimes of small misidentification probabilities. Applications include rapid epidemic detection (e.g., COVID-19), network intrusion monitoring, and tracking faint space objects. In particular, our methods identified regions of COVID outbreaks several days before official quarantine measures were imposed.

October 31: Yizhe Zhu (USC)

November 7: Yun Yang (University of Maryland, College Park)

Simulation-based Inference via Structured Score Matching

Simulation-based inference (SBI) provides an effective framework for statistical analysis when the likelihood is intractable but model simulations are available. This talk presents a unified SBI framework that leverages structured score matching for both frequentist and Bayesian inference. On the frequentist side, we develop a likelihood-free approach that combines score matching with gradient-based optimization and bootstrap procedures for parameter estimation and uncertainty quantification. On the Bayesian side, we integrate score matching with Langevin dynamics to efficiently explore complex posterior landscapes in moderate- to high-dimensional settings.  In both cases, we design tailored score-matching estimators and architectural regularizations that embed the statistical structure of log-likelihood scores, which in turn improves estimation accuracy and scalability.

November 14: Bixing Qiao (USC)

A New Approach for the Continuous Time Kyle-Back Strategic Insider Equilibrium Problem

Abstract: In this talk, we consider a continuous-time Kyle-Back model which is a game between an insider and a market maker. The existing literature typically focuses on constructing equilibria with a PDE approach, which requires certain Markovian structures. We characterize all equilibria through a coupled system of forward-backward SDEs. In particular, when the time duration is small, we show that the FBSDE is well-posed, and therefore the game has a unique equilibrium. Moreover, this unique equilibrium may be non-Markovian and thus not attainable via the PDE approach. We next study the set value of the game, which roughly speaking is the set of insider’s values over all equilibria and thus is by nature unique. Finally, we characterize the set value through a level set of a certain standard HJB equation.

In the second part of the talk, we apply the new approach to the Kyle-Back model with dynamic legal risk. In the presence of an active regulator and a large number of noise traders, the insider chooses a strategy that conceals his identity within a large volume of surrounding trades and concentrates on medium-sized trades. We establish an intensity-based mathematical framework for explaining the interconnections between insider trading and stealth trading. It turns out that when the number of noise traders becomes large, the price impact of the insider asymptotically vanishes, and consequently the stochastic game becomes deterministic optimizations, which also carry implications for regulatory investigations and sanctions.

The results above include the joint work with Weixuan Xia and with Jianfeng Zhang.

November 21: Xiaocong Xu (USC)

A leave-one-out approach to approximate message passing

Approximate message passing (AMP) has emerged both as a popular class of iterative algorithms and as a powerful analytic tool in a wide range of statistical estimation problems and statistical physics models. A well established line of AMP theory proves Gaussian approximations for the empirical distributions of the AMP iterate in the high dimensional limit, under the GOE random matrix model and other rotational invariant ensembles. This work provides a non-asymptotic, leave-one-out representation for the AMP iterate that holds under a broad class of Gaussian random matrix models with general variance profiles. In contrast to the typical AMP theory that describes the first-order behavior for the empirical distributions of the AMP iterate via a low dimensional state evolution, our leave-one-out representation yields an intrinsically high dimensional state evolution formula, which provides a second-order, non-asymptotic characterization for the possibly heterogeneous, entrywise behavior of the AMP iterate under the prescribed random matrix models. To exemplify some distinct features of our AMP theory in applications, we analyze, in the context of regularized linear estimation, the precise stochastic behavior of the Ridge estimator for independent and non-identically distributed observations whose covariates exhibit general variance profiles. We find that its finite-sample distribution is characterized via a weighted Ridge estimator in a heterogeneous Gaussian sequence model. Notably, in contrast to the i.i.d. sampling scenario, the effective noise and regularization are now full dimensional vectors determined via a high dimensional system of equations. Our leave-one-out method of proof differs significantly from the widely adopted conditioning approach for rotational invariant ensembles, and relies instead on an inductive method that utilizes almost solely integration-by-parts and concentration techniques. This talk is based on joint work with Zhigang Bao and Qiyang Han.

November 28: [no classes]

December 5: Zebin Wang (Harvard)

DANIEL: A Distributed and Scalable Approach for Global Representation Learning with EHR Applications

Classical probabilistic graphical models face fundamental challenges in modern data environments, which are characterized by high dimensionality, source heterogeneity, and stringent data-sharing constraints. In this work, we revisit the Ising model, a well-established member of the Markov Random Field (MRF) family, and develop a distributed framework that enables scalable and privacy-preserving representation learning from large-scale binary data with inherent low-rank structure. Our approach optimizes a non-convex surrogate loss function via bi-factored gradient descent, offering substantial computational and communication advantages over conventional convex approaches. We evaluate our algorithm on multi-institutional electronic health record (EHR) datasets from 58,248 patients across the University of Pittsburgh Medical Center (UPMC) and Mass General Brigham (MGB), demonstrating superior performance in global representation learning and downstream clinical tasks, including relationship detection, patient phenotyping, and patient clustering. These results highlight a broader potential for statistical inference in federated, high-dimensional settings while addressing the practical challenges of data complexity and multi-institutional integration.

Spring 2025:

January 17: Bradley Rava (University of Sydney’s Business School)

Ask for More Than Bayes Optimal: A Theory of Indecisions for Classification

February 14 : Po-Ling Loh (University of Cambridge)

Differentially private M-estimation via noisy optimization

February 21: Lijun Ding (UCSD)

Flat minima generalize for low-rank matrix recovery

Empirical evidence suggests that for a variety of overparameterized nonlinear models, most notably in neural network training, the growth of the loss around a minimizer strongly impacts its performance. Flat minima — those around which the loss grows slowly — appear to generalize well. This work takes a step towards understanding this phenomenon by focusing on the simplest class of overparameterized nonlinear models: those arising in low-rank matrix recovery. We analyze overparameterized matrix and bilinear sensing, robust PCA, covariance matrix estimation, and single hidden layer neural networks with quadratic activation functions. In all cases, we show that flat minima, measured by the trace of the Hessian, exactly recover the ground truth under standard statistical assumptions. For matrix completion, we establish weak recovery, although empirical evidence suggests exact recovery holds here as well.

February 28: Nils Detering (Heinrich-Heine University)

joint with Math Finance Colloquium

A class of point-wise operating SPDE coefficients for HJM models

We present a dynamic model for forward curves within the Heath-Jarrow-Morton framework under the Musiela parametrization. The forward curves take values in a function space $\h$, and their dynamics follows a stochastic partial differential equation with state-dependent coefficients. In particular, the coefficients are defined through point-wise operating maps on $\h$, resulting in a locally state-dependent structure. We first explore conditions under which these point-wise operators are well defined on $\h$. Next, we determine conditions to ensure that the resulting coefficient functions satisfy local growth and Lipschitz properties, so to guarantee the existence and uniqueness of mild solutions. The proposed model captures the behavior of the entire forward curve through a single equation, yet retains remarkable simplicity. Notably, we demonstrate that certain one-dimensional projections of the model are Markovian and satisfy a one-dimensional stochastic differential equation. This connects our Hilbert-space approach to well established models for forward contracts with fixed delivery times, for which existing formulas and numerical techniques can be applied. This link allows us to examine also conditions for maintaining positivity of the solutions. As concrete examples, we analyze Hilbert-space valued variants of an exponential model and of a constant elasticity of variance model.

March 7: Pedro Abdalla Teixeira (UCI)

Covariance Estimation through Empirical Process Theory

In this talk, we revisit the classical problem of covariance estimation from the perspective of empirical process theory. In particular, we explore the relationship between various covariance estimation problems and their associated empirical processes. This talk is based on a series of works by the author in collaboration with Nikita Zhivotovskiy and Shahar Mendelson.

March 28: Nikita Zhivotovskiy (UC Berkeley)

Sharper Risk Bounds for Statistical Aggregation

In this talk, we revisit classical results in the theory of statistical aggregation, focusing on the transition from global complexity to a more manageable local one. The goal of aggregation is to combine several base predictors to achieve a prediction nearly as accurate as the best one, without assumptions on the class structure or target. Though studied in both sequential and statistical settings, they traditionally use the same “global” complexity measure. We highlight the lesser-known PAC-Bayes localization enabling us to prove a localized bound for the exponential weights estimator by Leung and Barron, and a deviation-optimal localized bound for Q-aggregation. Finally, we demonstrate that our improvements allow us to obtain bounds based on the number of near-optimal functions in the class, and achieve polynomial improvements in sample size in certain nonparametric situations. This is contrary to the common belief that localization doesn’t benefit nonparametric classes. Joint work with Jaouad Mourtada and Tomas Vaškevičius.

April 4: Bohan Zhou (UCSB)

The Generalized Wasserstein Barycenter Problem

The Wasserstein barycenter problem, introduced by Agueh and Carlier, has gained widespread interest in machine learning, statistics, computer graphics, and engineering. Given the prominence of the Wasserstein distance, computing its barycenter is often a foundational step in understanding the properties of probability spaces equipped with this metric. In this talk, we explore the generalized Wasserstein barycenter, allowing for negative weights. This extension naturally arises when transitioning from interpolation to extrapolation, and it has immediate implications for high-order schemes in gradient flow (by Han, Esedoglu, and Garikipati), adversarial multiclass classification (by Garcia Trillos, Jacobs, and Kim), and Wasserstein Regression (by Chen, Lin and Muller)  We will discuss the theoretical results (existence, optimality condition and uniqueness) of the problem, analyze special cases where all weights are positive except one or all are negative except one, and introduce new numerical algorithms applicable to both generalized and classical settings, concluding with experimental results.

April 11: Abiy Tasissa (Tufts)

From missing distances to structures: Theory, algorithms and applications

The advancement of technology has significantly enhanced our capacity to collect data. However, in many real-world applications, certain inherent limitations—such as the precision of measurement devices, environmental conditions, or operating costs—can result in missing data. In this talk, we focus on the setting where the available data consist of pairwise distances between a set of points, with the goal of estimating the configuration of the underlying points from incomplete distance measurements. This is known as the Euclidean distance geometry (EDG) problem and is central to many applications.

We first start by describing the solution when all distances are given using the classical multidimensional scaling (MDS) technique and then discuss a constructive approach to interpret the key mathematical objects in MDS. Next, we introduce a mathematical framework to address the EDG problem under two sampling models of the distance matrix: global sampling (uniform sampling of the entries of the distance matrix) and structured local sampling, where the measurements are limited to a subset of rows and columns. We discuss the conditions required for the exact recovery of the point configuration and the associated algorithms. The last part of the talk will illustrate the algorithms using synthetic and real data and discuss ongoing work.

Fall 2024:

August 30: Yiyun He (UC Irvine)

Differentially Private Algorithms for Synthetic Data

September 6: Omer Tamuz (Caltech)

Asymptotic Renyi Entropies of Random Walks on Groups

We introduce asymptotic Renyi entropies as a parameterized family of invariants for random walks on groups. These invariants interpolate between various well-studied properties  of the random walk, including the growth rate of the group, the Shannon entropy, and the spectral radius. They furthermore offer large deviation counterparts  of the Shannon-McMillan-Breiman Theorem. We prove some basic properties of asymptotic Renyi entropies that apply to all groups, and discuss their analyticity and positivity for the free group and lamplighter groups.

Joint with Kimberly Golubeva and Minghao Pan

September 20: Weixin Yao (UC Riverside)

October 4: Richard Y. Zhang (UIUC)

Rank Overparameterization and Global Optimality Certification for Large-Scale Low-rank Optimization

Numerous important problems across applied statistics reduce into nonconvex estimation / optimization over a low-rank matrix. In principle, these can be reliably solved to global optimality via convex relaxation, but the computational costs can become prohibitive on a large scale. In practice, it is much more common to optimize over the low-rank matrices directly, as in the Burer-Monteiro approach, but their nonconvexity can cause failure by getting stuck at a spurious local minimum. For safety-critical societal applications, such as the operation and planning of an electricity grid, our inability to reliably achieve global optimality can have significant real-world consequences.

In the first part of this talk, we investigate how overparameterizing the low-rank factorization can render its nonconvexity increasingly benign. In the smooth and strongly-convex setting, we rigorously show that, as the rank is increased, spurious local minima become increasingly rare in a step-wise fashion. In other words, rank-2 has fewer spurious local minima than rank-1, and rank-3 has fewer than rank-2, etc.  Once the rank exceeds an O(1) threshold, every remaining local minimum is a global minimum, and every saddle point can be escaped. In the second part of this talk, we use the rank deficiency brought on by rank overparameterization to certify convergence to global optimality after the fact. The certification is an a posteriori guarantee that is valid under much weaker assumptions than typical “no spurious local minima” guarantees. However, rank deficiency significantly slows down the convergence of gradient descent, from a linear rate to a sublinear rate. We propose an inexpensive preconditioner that restores the convergence rate of gradient descent back to linear in the overparameterized case.

Main related papers:
https://arxiv.org/abs/2207.01789
https://arxiv.org/abs/2206.03345 (joint work with Gavin Zhang and Salar Fattahi)

October 11: [no classes]

October 18: Camilo Hernández (USC ISE)

The mean field Schrödinger problem: a mean field control perspective

The mean field Schrödinger problem (MFSP) is the problem of finding the most likely path of a McKean-Vlasov type particle with constrained initial and final configurations. It was first introduced by Backhoff et al. (2020), who studied its existence and long-time behavior. This talk aims to show how ideas from mean field control theory allow us to derive new interesting results on the MFSP. In particular, we study its existence, characterization, and the so-called convergence problem. The method rests upon studying suitably penalized problems and stochastic control techniques. This talk is based on a joint work with Ludovic Tangpi (Princeton).

October 25: Yizhe Zhu (USC)

Non-convex matrix sensing: Breaking the quadratic rank barrier in the sample complexity

For the problem of reconstructing a low-rank matrix from a few linear measurements, two classes of algorithms have been widely studied in the literature: convex approaches based on nuclear norm minimization, and non-convex approaches that use factorized gradient descent. Under certain statistical model assumptions, it is known that nuclear norm minimization recovers the ground truth as soon as the number of samples scales linearly with the number of degrees of freedom of the ground truth. In contrast, while non-convex approaches are computationally less expensive, existing recovery guarantees assume that the number of samples scales at least quadratically with the rank. In this talk, we consider the problem of reconstructing a positive semidefinite matrix from a few Gaussian measurements. We improve the previous rank-dependence in the sample complexity of non-convex matrix factorization from quadratic to linear. Our proof relies on a probabilistic decoupling argument, where we show that the gradient descent iterates are only weakly dependent on the individual entries of the measurement matrices. Joint work with Dominik Stöger (KU Eichstätt-Ingolstadt).

November 1: Wojciech Ozanski (Florida State University)
[Joint with PDE seminar]

Instantaneous continuous loss of regularity for the SQG equation

November 8: Johannes Wiesel (CMU)
[Joint with Mathematical Finance seminar]

Bounding adapted Wasserstein metrics

The Wasserstein distance $\mathcal{W}_p$ is an important instance of an optimal transport cost. Its numerous mathematical properties as well as applications to various fields such as mathematical finance and statistics have been well studied in recent years. The adapted Wasserstein distance $\mathcal{A}\mathcal{W}_p$ extends this theory to laws of discrete time stochastic processes in their natural filtrations, making it particularly well suited for analyzing time-dependent stochastic optimization problems.

While the topological differences between $\mathcal{A}\mathcal{W}_p$ and $\mathcal{W}_p$ are well understood, their differences as metrics remain largely unexplored beyond the trivial bound $\mathcal{W}_p\lesssim \mathcal{A}\mathcal{W}_p$. This paper closes this gap by providing upper bounds of $\mathcal{A}\mathcal{W}_p$ in terms of $\mathcal{W}_p$ through investigation of the smooth adapted Wasserstein distance. Our upper bounds are explicit and are given by a sum of $\mathcal{W}_p$, Eder’s modulus of continuity and a term characterizing the tail behavior of measures. As a consequence, upper bounds on $\mathcal{W}_p$ automatically hold for $\mathcal{AW}_p$ under mild regularity assumptions on the measures considered. A particular instance of our findings is the inequality $\mathcal{A}\mathcal{W}_1\le C\sqrt{\mathcal{W}_1}$ on the set of measures that have Lipschitz kernels.

Our work also reveals how smoothing of measures affects the adapted weak topology. In fact, we find that the topology induced by the smooth adapted Wasserstein distance exhibits a non-trivial interpolation property, which we characterize explicitly: it lies in between the adapted weak topology and the weak topology, and the inclusion is governed by the decay of the smoothing parameter.

This talk is based on joint work with Jose Blanchet, Martin Larsson and Jonghwa Park.

November 15: Greta Panova (USC)

Algebra meets probability: permutons from pipe dreams via integrable probability

November 22: Gil Kur (ETH Zürich)  

Note the non-standard time and location: 1pm in KAP 265

Minimum Norm Interpolation Meets The Local Theory of Banach Spaces

Minimum-norm interpolators have recently gained attention as an analyzable model to shed light on the double descent phenomenon observed for neural networks.  Most of the work has focused on analyzing interpolators in Hilbert spaces, where, typically, an effectively low-rank structure of the feature covariance prevents a large bias. This work takes a first step towards establishing a general framework that connects generalization properties of the interpolators to well-known concepts from high-dimensional geometry, specifically, from the local theory of Banach spaces.

November 29: [no classes]

December 6: Elliot Paquette (McGill) [2PM! Nonstandard time!]

Random matrix theory for high dimensional optimization, and an application to scaling laws

We describe a program of analysis of stochastic gradient methods on high-dimensional random objectives.  We illustrate some assumptions under which the loss curves are universal, in that they can completely be described in terms of some underlying covariance structure of the problem setup.   Furthermore, we give a description of these loss curves that can be analyzed precisely. 

As a motivating application, we show how this can be applied to the power-law-random-features model.  This is a simple two-hyperparameter family of optimization problems, which displays 5 distinct phases of SGD loss curves; these phases are determined by the relative complexities of the target, data distribution, and whether these are ‘high-dimensional’ or not (which in context can be precisely defined).  In each phase, we can also give, for a given compute budget, the optimal random-feature dimensionality.

Joint work with Courtney Paquette (McGill & Google Deepmind), Jeffrey Pennington (Google Deepmind), and Lechao Xiao  (Google Deepmind).