View abstract

Session I.6 - Mathematical Foundations of Data Assimilation and Inverse Problems

Wednesday, June 14, 16:30 ~ 17:00

Sampling with constraints

Xin Tong

National University of Singapore, Singapore   -   This email address is being protected from spambots. You need JavaScript enabled to view it.

Sampling-based inference and learning techniques, especially Bayesian inference, provide an essential approach to handling uncertainty in machine learning (ML).   As these techniques are increasingly used in daily life, it becomes essential to safeguard the ML systems with various trustworthy-related constraints, such as fairness, safety, interpretability. We propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint or a level set  specified by a general nonlinear function. By exploiting the gradient flow structure of LD and SVGD, we derive algorithms for handling constraints, including a  primal-dual gradient approach and the constraint controlled gradient descent approach. 
We investigate the continuous-time mean-field limit of these algorithms and show that they have $O(1/t)$ convergence under mild conditions.  

Joint work with Qiang Liu (UT-Austin), Xingchao Liu (UT-Austin) and Ruqi Zhang (Purdue).

View abstract PDF