Session I.6 - Mathematical Foundations of Data Assimilation and Inverse Problems
Talks
Monday, June 12, 14:00 ~ 14:30
Gradient-based dimension reduction for solving Bayesian inverse problems
Ricardo Baptista
California Institute of Technology, USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
Computational Bayesian inference aims to characterize the posterior probability distributions for parameters in statistical models. The complexity of many inference methods such as MCMC and variational inference, however, typically scale poorly with the growing dimensions of model parameters and data. A recent approach to deal with high or possibly even infinite-dimensional parameters is to exploit low-dimensional structure in the inverse problem and approximately reformulate it in low-to-moderate dimensions. In this presentation, we will introduce an information-theoretic analysis to bound the error from reducing the dimensions of both parameters and data. This bound exploits gradient evaluations of the log-likelihood function to identify relevant low-dimensional subspaces for these variables as well as reveal reduced dimensions that result in minimal error. The benefit of the proposed dimension reduction technique will be demonstrated using several inference algorithms on applications including image processing and data assimilation for aerodynamic flows.
Joint work with Youssef Marzouk (Massachusetts Institute of Technology, USA) and Olivier Zahm (INRIA Grenoble Alpes, France).
Monday, June 12, 14:30 ~ 15:00
Higher Order Ensemble Kalman Filtering
Tyrus Berry
George Mason University, USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
In data assimilation, as in solving more general inverse problems, a key decision is how to represent the prior and posterior distributions. Methods such as the Kalman filter, extended Kalman filter, and variational filters work with a parametric family of distributions, whereas particle filters represent the distribution non-parametrically with a weighted set of representative samples. Ensemble transform methods such as the ensemble Kalman filter (EnKF), try to find a middle ground between the parametric and nonparametric extremes by using an ensemble of particles that is optimally chosen based on a parametric family. For example, the Unscented Kalman Filter (UKF) chooses optimal cubature nodes for a Gaussian approximation to the distribution. This can also be shown to be the maximum entropy approximation which matches the first two moments of the distribution. In this talk we generalize this approach to higher moments, (e.g. skewness, kurtosis, etc.) using a novel computationally feasible approximation to the CANDECOMP/PARAFAC (CP) tensor decomposition. This allows us to represent a distribution using a small ensemble of particles that capture the first four moments of the distribution (and generalizes to higher moments). Using rigorous error bounds, we can show weak convergence to the true distribution in the limit as the number of moments tends to infinity, a convergence result analogous to those of particle methods which take a limit as the number of particles tends to infinity. Finally, motivated by our higher order ensemble transform, we derive a higher order generalization of the Kalman equations based on a maximal entropy closure, generalizing the classical approach to these higher moments.
Joint work with Deanna Easley.
Monday, June 12, 15:00 ~ 15:30
Bayesian online algorithms for learning data-driven models of chaotic dynamics
Marc Bocquet
CEREA, École des Ponts and EdF R&D, Île-de-France, France - This email address is being protected from spambots. You need JavaScript enabled to view it.
The advent of machine and more specifically deep learning techniques significantly boosts the capabilities of data assimilation and inverse problem techniques used in the geosciences. It also spurs new, more ambitious goals for data assimilation in high dimensions. One of the key, currently very popular, area of research consists in learning data-driven models of dynamical systems. With the natural constraints of geoscience, i.e., sparse and noisy observations, this typically requires the joint use of data assimilation and neural networks. However, the vast majority of algorithms are offline; they rely on a set of observations from the physical system, which must be available before the start of the training.
We propose new algorithms that update the knowledge of the surrogate (i.e., data-driven) model when new observations become available. We carry out this objective with both variational (weak-constraint 4D-Var like) and ensemble (EnKF and IEnKS like) techniques. We test these algorithms on low-order Lorenz models, on quasi-geostrophic models, the ERA5 dataset, and sea-ice dynamics. Remarkably, in several cases, the online algorithms significantly outperform the offline ones. This opens the way to adaptive surrogate models that progressively learn trends and conform to real-time constraints of operational weather forecasting.
Monday, June 12, 15:30 ~ 16:00
Particle dynamics for rare event estimation with PDE-based models
Elisabeth Ullmann
Technical University of Munich, Germany - This email address is being protected from spambots. You need JavaScript enabled to view it.
The estimation of the probability of rare events is an important task in reliability and risk assessment of critical societal systems, for example, groundwater flow and transport, and engineering structures. In this talk we consider rare events that are expressed in terms of a limit state function which depends on the solution of a partial differential equation (PDE). We present two novel estimators for the rare event probability based on (1) the Ensemble Kalman filter for inverse problems, and (2) a consensus-building mechanism. Both approaches use particles which follow a suitable stochastic dynamics to reach the failure states. The particle methods have historically been used for Bayesian inverse problems. We connect them to rare event estimation.
Joint work with Konstantin Althaus, Fabian Wagner and Iason Papaioannou (TUM).
Monday, June 12, 16:30 ~ 17:30
Nonlinear manifold approximations for reduced-order modeling of nonlinear systems
Karen Willcox
The University of Texas at Austin, USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
The majority of existing reduced-order modeling methods use a linear subspace for dimensionality reduction. This has the significant mathematical advantage of leading to a reduced model with known and analyzable structure; however, for complex systems and transport-dominated dynamics, linear compression often does not yield a sufficiently rich approximation. This talk presents our recent work in using nonlinear dimensionality reduction via quadratic manifolds combined with our non-intrusive Operator Inference approach. The approach can be viewed as a form of data-driven closure modeling, since the quadratic component introduces directions into the approximation that lie in the orthogonal complement of the linear subspace, but without introducing any additional degrees of freedom to the low-dimensional representation. The result is reduced-order models that benefit from the increased richness of the representation in a quadratic manifold, while retaining an analyzable structure.
Joint work with Rudy Geelen (The University of Texas at Austin), Steve Wright (University of Wisconsin) and Laura Balzano (University of Michigan).
Monday, June 12, 17:30 ~ 18:00
Sequential Bayesian Learning
Jana de Wiljes
Uni Potsdam, Germany - This email address is being protected from spambots. You need JavaScript enabled to view it.
In various application areas it is crucial to make predictions or decisions based on sequentially incoming observations and previous existing knowledge on the system of interest. The prior knowledge is often given in the form of evolution equations (e.g., ODEs derived via first principles or fitted based on previously collected data), from here on referred to as model. Despite the available observation and prior model information, accurate predictions of the "true" reference dynamics can be very difficult. Common reasons that make this problem so challenging are: (i) the underlying system is extremely complex (e.g., highly nonlinear) and chaotic (i.e., crucially dependent on the initial conditions), (ii) the associate state and/or parameter space is very high dimensional (e.g., worst case $10^8$), (iii) observations are noisy, partial in space and discrete in time. In practice these obstacles are combated with a series of approximations (the most important ones being based on assuming Gaussian densities and using Monte Carlo type estimations) and numerical tools that work surprisingly well in some settings. Yet the mathematical understanding of the signal tracking ability of a lot of these methods is still lacking. Additionally, solutions of some of the more complicated problems that require simultaneous state and parameter estimation (including control parameters that can be understood as decisions/actions performed) can still not be approximated in a computationally feasible fashion. Here we will discuss how techniques from the world of machine learning can aid to overcome some of the computational challenges.
Monday, June 12, 18:00 ~ 18:30
Combining Machine Learning and Stochastic Methods for Modeling and Forecasting Complex Systems
Georg Gottwald
University of Sydney, Australia - This email address is being protected from spambots. You need JavaScript enabled to view it.
Random feature maps can be viewed as a single hidden layer network in which the weights of the hidden layer are fixed and only those of the outer layer are learned. We show that random feature maps allow for sequential learning when combined with an ensemble Kalman filter, leading to improved forecasts when compared to standard random feature map learning. The method can be extended to the case of noisy partial observations. Random feature maps, we show, can further be used to learn subgridscale parametrizations from noisy data as well as produce reliable ensembles.
Joint work with Sebastian Reich (Universität Potsdam).
Tuesday, June 13, 14:00 ~ 15:00
Some theoretical aspects of Particle Filters and Ensemble Kalman Filters
Pierre Del Moral
INRIA, France - This email address is being protected from spambots. You need JavaScript enabled to view it.
In the last three decades, Particle Filters (PF) and Ensemble Kalman Filters (EnKF) have become one of the main numerical techniques in data assimilation, Bayesian statistical inference and nonlinear filtering.
Both particle algorithms can be interpreted as mean field type particle interpretation of the filtering equation and the Kalman recursion. In contrast with conventional particle filters, the EnKF is defined by a system of particles evolving as the signal in some state space with an interaction function that depends on the sample covariance matrices of the system.
Despite widespread usage, little is known about the mathematical foundations of EnKF. Most of the literature on EnKF amounts to designing different classes of useable observer-type particle methods. To design any type of consistent and meaningful filter, it is crucial to understand their mathematical foundations and their learning/tracking capabilities. This talk discusses some theoretical aspects of these numerical techniques. We present some recent advances on the stability properties of these filters. We also initiate a comparison between these particle samplers and discuss some open research questions.
Tuesday, June 13, 15:00 ~ 15:30
The Ensemble Kalman Filter in the Near-Gaussian Setting
Franca Hoffmann
California Institute of Technology, United States - This email address is being protected from spambots. You need JavaScript enabled to view it.
We provide analysis for the accuracy of the ensemble Kalman filter for problems where the filtering distribution is non-Gaussian, but can be characterized as close to Gaussian after appropriate lifting to the joint space of state and data. The ensemble Kalman filter is widely used in applications because, for high dimensional filtering problems, it has a robustness that is not shared for example by the particle filter; in particular it does not suffer from weight collapse. However, there is no theory which quantifies its accuracy, as an approximation of the true filtering distribution, except in the Gaussian setting. We use the mean-field description to address this issue. Our results rely on stability estimates that can be obtained by rewriting the mean field ensemble Kalman filter in terms of maps on probability measures, and then introducing a weighted total variation metric in which these maps are locally Lipschitz.
Joint work with Jose Antonio Carrillo (University of Oxford, United Kingdom), Andrew M. Stuart (California Institute of Technology, United States) and Urbain Vaes (INRIA Paris, France).
Tuesday, June 13, 15:30 ~ 16:00
Optimal Transport Particle Filters
Bamdad Hosseini
University of Washington, USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
Filtering of high dimensional and nonlinear models, with highly non-Gaussian states, is a challenging problem where traditional filtering algorithms such as EnKF fail. In this talk, I will discuss some new ideas and approaches to this problem using optimal transport theory and triangular maps leading to interesting theoretical observations and a path towards scalable algorithms.
Joint work with Amirhossein Taghvaei (University of Washington, USA) and Mohammad Al-Jarrah (University of Washington, USA).
Tuesday, June 13, 16:30 ~ 17:00
Stability of the nonlinear filter against prior knowledge via duality formalism
Jin Won Kim
University of Potsdam, Germany - This email address is being protected from spambots. You need JavaScript enabled to view it.
My talk is concerned with the asymptotic stability analysis of the nonlinear filter. The analysis is closely based on duality between estimation problem and the optimal control problem.
The stability analysis of the Kalman filter relies, either direct or indirect manner, on duality theory: the asymptotic stability of the Kalman filter is related to the asymptotic stability of the (dual) optimal control system, and therefore the necessary and sufficient conditions for the Kalman filter is given by the controllability of the control system, which is equivalent to the detectability of the model due to the duality.
Meanwhile, it has been considered that the duality theory is an artifact of linear Gaussian theory, and the duality theory had never been considered for both in t he definitions of stochastic observability and in the stochastic filter stability analysis. In my talk, I will relate the observability and ergodicity with the (dual) optimal control problem, and propose a slightly strengthened assumption to achieve filter stability property.
Tuesday, June 13, 17:00 ~ 17:30
Learning linear operators: infinite-dimensional regression as an inverse problem
Mattes Mollenhauer
Freie Universität Berlin, Germany - This email address is being protected from spambots. You need JavaScript enabled to view it.
We consider the problem of learning a linear operator between two Hilbert spaces from empirical observations, which we interpret as least squares regression in infinite dimensions. We show that this goal can be reformulated as a statistical inverse problem with unknown noncompact forward operator. However, we prove that, in terms of spectral properties, this problem is equivalent to the well-known compact inverse problem with scalar response regression. Our framework allows for the elegant derivation of dimension-free rates for generic learning algorithms under Hölder-type source conditions. The rates holds for a variety of relevant scenarios in functional regression and nonparametric regression with operator-valued kernels and match those of classical kernel regression with scalar response.
The preprint is available under https://arxiv.org/abs/2211.08875
Joint work with Nicole Mücke (TU Braunschweig, Germany) and Tim Sullivan (U Warwick, UK).
Tuesday, June 13, 17:30 ~ 18:00
An involution framework for Metropolis-Hastings algorithms on general state spaces
Cecilia Mondaini
Drexel University, USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
We consider a general framework for Metropolis-Hastings algorithms used to sample from a given target distribution on general state spaces. Our framework is based on a fundamental involution structure, and shown to encompass several popular algorithms as special cases, both in the finite- and infinite-dimensional settings. In particular, random walk, preconditioned Crank-Nicolson (pCN), schemes based on a suitable Langevin dynamics such as the Metropolis Adjusted Langevin algorithm (MALA), and also ones based on Hamiltonian dynamics including several variants of the Hamiltonian Monte Carlo (HMC) algorithm. In addition, we provide an abstract framework for algorithms that generate multiple proposals at each iteration, which yield efficient sampling schemes through the use of modern parallel computing resources. Here we derive several generalizations of the aforementioned algorithms following as special cases of this multiproposal framework. To illustrate effectiveness of these sampling procedures, we present applications in the context of some Bayesian inverse problems in fluid dynamics.
Joint work with Nathan Glatt-Holtz (Tulane University), Andrew Holbrook (UCLA) and Justin Krometis (Virginia Tech).
Tuesday, June 13, 18:00 ~ 18:30
Markov chain Monte Carlo and high-dimensional, nonlinear inverse problems in Earth Science
Matthias Morzfeld
Scripps Institution of Oceanography, University of California, San Diego, USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
Earth science nearly always requires estimating models, or model parameters, from data. This could mean to infer the state of the southern ocean from ARGO floats, to compute the state of our atmosphere based on atmospheric observations of the past six hours, or to construct a resistivity model of the Earth’s subsurface from electromagnetic data. All these problems have in common that the number of unknowns is large (millions to hundreds of millions) and that the underlying processes are nonlinear. The problems also all have in common that they can be formulated as the problem of drawing samples from a high-dimensional Bayesian posterior distribution.
Due to the nonlinearity, Markov chain Monte Carlo (MCMC) is a good candidate for the numerical solution of geophysical inverse problems. But MCMC is known to be slow when the number of unknowns is large. In this talk, I will argue that an unbiased solution of nonlinear, high-dimensional problems remains difficult, but one can construct efficient and accurate biased estimators that are feasible to apply to high-dimensional problems. I will show examples of biased estimators in action and invert electromagnetic data using an approximate MCMC sampling algorithm called the RTO-TKO (randomize-then-optimize -- technical-knock-out).
Joint work with Daniel Blatter (Lawrence Berkeley National Laboratory), Kerry Key (Lamont-Doherty Earth Observatory, Columbia University) and Steven Constable (Scripps Institution of Oceanography, University of California, San Diego).
Wednesday, June 14, 14:00 ~ 14:30
Hybrid ensemble data assimilation for hierarchical models
Dean Oliver
NORCE Norwegian Research Centre, Norway - This email address is being protected from spambots. You need JavaScript enabled to view it.
The choice of the prior model can have a large impact on the ability to assimilate data in geophysical inverse problems. In standard applications of ensemble Kalman-based data assimilation, all ensemble members from the prior are generated from the same prior covariance matrix. In a non-centered hierarchical approach, the parameters of the covariance function, that is the variance, the orientation of the anisotropy and the ranges in two principal directions, may all be uncertain. In this talk I discuss three approaches to sampling from the posterior for this type of problem: an optimization-based sampling approach, an iterative ensemble smoother (IES), and a hybrid of the previous two approaches (hybrid-IES). I apply the three methods to a linear sampling problem for which it is possible to compare results with marginal-then-conditional approach. I also test the IES and the hybrid-IES methods on a 2D flow problem with uncertain anisotropy in the prior covariance. The IES method is shown to perform poorly in the flow examples because of the poor representation of the local sensitivity matrix by the ensemble-based method. The hybrid method, however, samples well even with a much smaller ensemble size.
Wednesday, June 14, 14:30 ~ 15:00
Computational Challenges and Advancements in Edge-Preserving Methods for Dynamic and Large-Scale Data
Mirjeta Pasha
Tufts University, United States of America - This email address is being protected from spambots. You need JavaScript enabled to view it.
Fast-developing fields such as data science, uncertainty quantification, and machine learning rely on fast and accurate methods for inverse problems. Three emerging challenges on obtaining meaningful solutions to large-scale and data-intensive inverse problems are ill-posedness of the problem, large dimensionality of the parameters, and the complexity of the model constraints. Tackling the immediate challenges that arise from growing model complexities (spatiotemporal measurements) and data-intensive studies (large-scale and high-dimensional measurements), state-of-the-art methods can easily exceed their limits of applicability. In this talk we discuss recent advancements on edge-preserving and computationally efficient methods for computing solutions to dynamic inverse problems, where both the quantities of interest and the forward operator change at different time instances. In the first part of the talk, to remedy these difficulties, we apply efficient regularization methods that enforce simultaneous regularization in space and time (such as edge enhancement at each time instant and proximity at consecutive time instants) and achieve this with low computational cost and enhanced accuracy. In the remainder of the talk, we focus on designing spatio-temporal Bayesian Besov priors for computing the MAP and UQ estimate in large-scale and dynamic inverse problems. Numerical examples from a wide range of applications, such as tomographic reconstruction, image deblurring, and multichannel dynamic tomography are used to illustrate the effectiveness of the described methods.
Wednesday, June 14, 15:00 ~ 15:30
Alternatives to delta functions in Monte Carlo based Uncertainty Quantification
Sahani Pathiraja
University of New South Wales, Australia - This email address is being protected from spambots. You need JavaScript enabled to view it.
In many tasks such as uncertainty quantification, high dimensional integration and Bayesian inference, it is necessary to generate samples from some underlying distribution and/or compute expectation functionals. Monte Carlo or particle-based methods are a well-established, foundational tool for such tasks, largely due to their flexible simulation-based structure. This foundational idea has spurred the development of popular approaches including Ensemble Kalman methods, sequential Monte Carlo, Markov Chain Monte Carlo, control type particle filters and particle based variational inference. However, Monte Carlo based techniques are often inefficient in high dimensions, in representing distributional tails and non-Gaussian characteristics as well as in complex time-dependent systems. This is in part due to the reliance on points (i.e. delta functions) to approximate distributions. In this talk I will explore various sampling techniques that make use of alternatives to delta functions, e.g. the suite of Gaussian mixture type filters as well as kernel-based particle flow methods. Their performance characteristics will be examined both analytically and numerically compared to corresponding approaches that make use of standard Monte Carlo.
Wednesday, June 14, 15:30 ~ 16:00
Non-asymptotic analysis of ensemble Kalman updates: effective dimension and localization
Daniel Sanz-Alonso
University of Chicago, United States - This email address is being protected from spambots. You need JavaScript enabled to view it.
Many modern algorithms for inverse problems and data assimilation rely on ensemble Kalman updates to blend prior predictions with observed data. Ensemble Kalman methods often perform well with a small ensemble size, which is essential in applications where generating each particle is costly. In this talk I will introduce a non-asymptotic analysis of ensemble Kalman updates that rigorously explains why a small ensemble size suffices if the prior covariance has moderate effective dimension due to fast spectrum decay or approximate sparsity. I will present the theory in a unified framework, comparing several implementations of ensemble Kalman updates that use perturbed observations, square root filtering, and localization. As part of our analysis, we develop new dimension-free covariance estimation bounds for approximately sparse matrices that may be of independent interest.
Joint work with Omar Al Ghattas (University of Chicago).
Wednesday, June 14, 16:30 ~ 17:00
Sampling with constraints
Xin Tong
National University of Singapore, Singapore - This email address is being protected from spambots. You need JavaScript enabled to view it.
Sampling-based inference and learning techniques, especially Bayesian inference, provide an essential approach to handling uncertainty in machine learning (ML). As these techniques are increasingly used in daily life, it becomes essential to safeguard the ML systems with various trustworthy-related constraints, such as fairness, safety, interpretability. We propose a family of constrained sampling algorithms which generalize Langevin Dynamics (LD) and Stein Variational Gradient Descent (SVGD) to incorporate a moment constraint or a level set specified by a general nonlinear function. By exploiting the gradient flow structure of LD and SVGD, we derive algorithms for handling constraints, including a primal-dual gradient approach and the constraint controlled gradient descent approach. We investigate the continuous-time mean-field limit of these algorithms and show that they have $O(1/t)$ convergence under mild conditions.
Joint work with Qiang Liu (UT-Austin), Xingchao Liu (UT-Austin) and Ruqi Zhang (Purdue).
Wednesday, June 14, 17:00 ~ 17:30
DIRT: a tensorised inverse Rosenblatt transport method
Tiangang Cui
Monash University, Australia - This email address is being protected from spambots. You need JavaScript enabled to view it.
Characterising intractable high-dimensional random variables is one of the fundamental challenges in stochastic computation. It has broad applications in statistical physics, machine learning, uncertainty quantification, econometrics, and beyond. The recent surge of transport maps offers a mathematical foundation and new insights for tackling this challenge. In this talk, we will present a functional tensor-train (TT) based order-preserving construction of inverse Rosenblatt transport in high dimensions. It characterises intractable random variables via couplings with tractable reference random variables. By integrating our TT-based approach into a nested approximation framework inspired by deep neural networks, we are able to significantly expand its capability to random variables with complicated nonlinear interactions and concentrated density functions. We demonstrate the efficacy of the resulting deep inverse Rosenblatt transport (DIRT) on a range of applications in statistical learning and uncertainty quantification, including parameter estimation for dynamical systems, PDE-constrained inverse problems, and Bayesian filtering.
Joint work with Sergey Dolgov, Rob Scheichl, Olivier Zahm, and Yiran Zhao.
Wednesday, June 14, 17:30 ~ 18:00
Nonlinear Filtering and Smoothing for Very High-Dimensional Geophysical Systems
Peter Jan van Leeuwen
Colorado State University, United States - This email address is being protected from spambots. You need JavaScript enabled to view it.
Particle Flow Filters and Smoothers allow for sampling posterior probability density functions (pdf) in very high-dimensional spaces. They are based on iterative minimization of the KL-divergence (or other distance measures) between the pdf represented by the particles and the posterior pdf. The methodology can be seen as an ensemble of 3Dvars for a filter, and an ensemble of 4Dvars for a smoother, in which the particles interact during the minimization. We will discuss deterministic and stochastic versions of the Particle Flows, and their relative advantages. . We also present a surprisingly simple and robust solution to the problem that particle flows need the gradient of the log of the prior pdf, which is only know by its samples. The methodology uses straightforward kernel methods and localization procedures. finally, if time permits, we will discuss a continuous version of a particle flow smoother that avoids the need to consider the prior of the state, and only works with the prior of the errors in the model equations. All developments will be illustrated with low and high-dimensional geophysical examples.
Joint work with Chih-Chi Hu (Colorado State University and Cooperative Institute for Research in the Atmosphere, Fort Collins, USA).
Wednesday, June 14, 18:00 ~ 18:30
Statistical theory for transport-based generative modelling
Sven Wang
M.I.T., United States - This email address is being protected from spambots. You need JavaScript enabled to view it.
Measure transport provides a powerful toolbox for estimation and generative modelling of complicated probability distributions. The common principle is to learn a transport map between a simple reference distribution and a complicated target distribution. In this talk, we discuss recent advances in statistical guarantees for such methods. We discuss multiple relevant classes of maps: (1) triangular maps, which are the building blocks for 'autoregressive normalizing flows', (2) optimal transport maps and (3) ODE-based maps, where the coupling between reference and target is given by an ODE flow. This encompasses NeuralODEs, a popular method for generative modeling.
We derive non-asymptotic convergence rates for the distance between the transport-based estimator and the unknown 'ground truth' probability distribution, which converges to 0 algebraically in the statistical sample size. Our results imply that in certain cases, transport methods achieve minimax-optimal convergence rates for non-parametric density estimation, which was previously unknown.
Joint work with Youssef Marzouk (MIT, United States), Robert Ren (MIT, United States) and Jakob Zech (U Heidelberg, Germany).
Posters
Regression-Based Methods for Learning Dynamic Bayesian Networks
Sara Amato
Worcester Polytechnic Institute , United States - This email address is being protected from spambots. You need JavaScript enabled to view it.
Understanding interactions between variables in experimental data sets, as well as predicting information about these components at future times is of interest in many fields. Dynamic Bayesian Network (DBN) approaches address this problem. DBN analysis involves learning dependencies between variables (i.e. learning the parents of each node) and learning the values of parameters in the DBNs closed-form expression. It is assumed that the values of each random variable at any given time are independent, which allows for the parents of each node to be learned separately; however, in some real-world applications, this assumption may be too restrictive as the variables may interact with each other over time. Further, analyzing the learned topology to identify a subset of parameters which can be estimated from data is often ignored; however, performing this step is crucial to understand which relationships can be reliably uncovered and quantified. In this poster, we break the independence assumption and construct a closed-form expression for the DBN using a coupled system of regression equations with possible nonlinear terms. Additionally, we present a workflow to select identifiable parameters and estimate this subset using Markov Chain Monte Carlo (MCMC) sampling methods.
Joint work with Andrea Arnold (Worcester Polytechnic Institute).
Reduced Order Methods for Linear Gaussian Inverse Problems on separable Hilbert Spaces
Giuseppe Carere
University of Potsdam, Germany - This email address is being protected from spambots. You need JavaScript enabled to view it.
In Bayesian inverse problems, the computation of the posterior distribution can be computationally demanding, especially in many-query settings such as filtering, where a new posterior distribution must be computed many times. In this work we consider some computationally efficient approximations of the posterior distribution for linear Gaussian inverse problems defined on separable Hilbert spaces. We measure the quality of these approximations using the Kullback-Leibler divergence of the approximate posterior with respect to the true posterior and investigate their optimality properties. The approximation method exploits low dimensional behaviour of the update from prior to posterior, originating from a combination of prior smoothing, forward smoothing, measurement error and limited number of observations, analogous to the results of Spantini et al. [1] for finite dimensional parameter spaces. Since the data is only informative on a low dimensional subspace of the parameter space, the approximation class we consider for the posterior covariance consists of suitable low rank updates of the prior. In the Hilbert space setting, care must be taken when inverting covariance operators. We address this challenge by using the Feldman-Hajek theorem for Gaussian measures.
[1] Spantini, Alessio, Antti Solonen, Tiangang Cui, James Martin, Luis Tenorio, and Youssef Marzouk. “Optimal Low-Rank Approximations of Bayesian Linear Inverse Problems.” SIAM Journal on Scientific Computing 37, no. 6 (January 2015): A2451–87. https://doi.org/10.1137/140977308.
Joint work with Han Cheng Lie.
A score-based operator Newton construction of transport maps
Nisha Chandramoorthy
Georgia Institute of Technology, United States of America - This email address is being protected from spambots. You need JavaScript enabled to view it.
Transportation of probability measures underlies many core tasks in statistics and machine learning, from variational inference to generative modeling. A typical goal is to represent a target probability measure of interest as the pushforward of a tractable source measure through a learned map. We present a new construction of such a transport map, given the ability to evaluate the score of the target distribution. Specifically, we characterize the map as a zero of an infinite-dimensional score-residual operator and derive a Newton-type method for iteratively constructing a zero. We prove convergence of these iterations by invoking classical elliptic regularity theory for partial differential equations (PDE) and show that this construction enjoys rapid convergence, under smoothness assumptions on the target score. A key element of our approach is a generalization of the elementary Newton method to infinite-dimensional operators, other forms of which have appeared in nonlinear PDE and in dynamical systems. Our Newton construction, while developed in a functional setting, also suggests new iterative algorithms for approximating transport maps.
Joint work with Florian Schaefer (Georgia Institute of Technology, USA) and Youssef Marzouk (Massachusetts Institute of Technology, USA).
Stability for the optimal experimental design problem in Bayesian inverse problems
Duc-Lam Duong
LUT University, Finland - This email address is being protected from spambots. You need JavaScript enabled to view it.
We study some stability properties of the expected utility function in Bayesian optimal experimental design. We provide a framework for this problem in a non-parametric setting and prove a convergence rate of the expected utility with respect to a likelihood perturbation. This rate is uniform over the design space and its sharpness in the general setting is demonstrated by proving a lower bound in a special case. To make the problem more concrete we proceed by considering non-linear Bayesian inverse problems with Gaussian likelihood and verify that the assumptions set out for the general case are satisfied and regain the stability of the expected utility with respect to perturbations to the observation map. Theoretical convergence rates are demonstrated numerically in different examples.
Joint work with Tapio Helin (LUT University, Finland) and Rodrigo Rojo-Garcia (LUT University, Finland).
Towards analysis of localized ensemble Kalman-Bucy filters for convection dominated models and sparse observations
Gottfried Hastermann
University of Potsdam, Germany - This email address is being protected from spambots. You need JavaScript enabled to view it.
With large scale availability of precise real time data, their incorporation into physically based predictive models, became increasingly important. This procedure of combining the prediction and observation is called data assimilation. A quite popular family of algorithms for this task are the ensemble variants of the Kalman filter.
In this work, we consider application and analysis of the localized ensemble Kalman-Bucy filter to (non-linear) models from space weather and fluid dynamics. In both cases, the analytical model state space is high or infinite dimensional, but only sparse observations are available. Therefore, a localized covariance operator, will be crucial in practice as well as for the analysis of a priori estimates on the empirical mean in the finite particle approximation.
Joint work with Jana de Wiljes (University of Potsdam, Germany).
Machine Learning for Missing Dynamics
Shixiao Jiang
ShanghaiTech University, China - This email address is being protected from spambots. You need JavaScript enabled to view it.
In this work, we present a general framework for recovering missing dynamical systems using available data and machine learning techniques. The proposed framework reformulates the prediction problem as a supervised learning problem to approximate a map that takes the memories of the resolved and identifiable unresolved variables to the missing components in the resolved dynamics. The map for this non-Markovian transition kernel is represented by a conditional distribution which is estimated from appropriate RKHS formulation or the long short term memory (LSTM). In the case of short memory terms or Gaussian variables, the success of the RKHS formulation suggests that various parametric modeling approaches that were proposed in various domain of applications can be understood through our RKHS representations. In the case of long-memory non-Markovian terms with non-Gaussian distribution, the LSTM method is an effective tool for recovering the missing dynamics that involves approximation of high-dimensional functions. Supporting numerical results on instructive nonlinear dynamics, including the two-layer Lorenz system, the truncated Burger-Hopf equation, the 57-mode barotropic stress model, and the Kuramoto-Sivashinsky (KS) equation.
Joint work with John Harlim (Penn State University), Haizhao Yang (Purdue University) and Senwei Liang (UC Berkeley).
Data assimilation for gas pipeline flow using observers based on velocity measurements
Teresa Kunkel
Technische Universität Darmstadt, Germany - This email address is being protected from spambots. You need JavaScript enabled to view it.
Our goal is to estimate the system state of gas flowing through gas pipes from distributed measurements of the velocity. Therefore we construct a Luenberger-type observer system that combines these measurements with the one-dimensional barotropic Euler equations as a model of gas flow, which we complement with energy-consistent coupling conditions at pipe junctions. First, we show the existence of Lipschitz-continuous, semi-global solutions of the observer system and of the original system on general networks under the assumption that both posses smooth initial and boundary data satisfying suitable smallness and compatibility conditions. Then, based on a modification of the relative energy method we show that the state of the observer system converges exponentially in the long time limit to the state of the original system, i.e., we reconstruct the complete system state using measurements of only one state variable. This result can be shown for a single pipe and for star-shaped networks.
Joint work with Jan Giesselmann (Technische Universität Darmstadt, Germany) and Martin Gugat (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany).
EKI with dropout
Shuigen Liu
National University of Singapore, Singapore - This email address is being protected from spambots. You need JavaScript enabled to view it.
Ensemble Kalman inversion (EKI) is an ensemble-based method to solve inverse problems. However, EKI can face difficulties when dealing with high-dimensional problems using a fixed-size ensemble, due to its subspace property where the ensemble always lives in the subspace spanned by the initial ensemble. To address this issue, we propose a novel approach using dropout regularization to mitigate the subspace problem. Compared to the conventional localization approach, dropout preserves the affine invariant property and avoids complex designing in the localization process. We prove that EKI with dropout converges with small ensemble settings, and the complexity of the algorithm scales linearly with dimension. Numerical examples demonstrate the effectiveness of our approach.
Joint work with Xin T. Tong (National University of Singapore, Singapore) and Sebastian Reich (Potsdam University, Germany).
On continuous-time Bayesian inference and the geometry of the probability manifold
Aimee Maurais
Massachusetts Institute of Technology, United States of America - This email address is being protected from spambots. You need JavaScript enabled to view it.
Inference in the Bayesian setting can be viewed as the problem of transforming a prior probability measure into a posterior measure. This transformation is frequently performed in “one shot” by applying a single update to an empirical or parametric approximation of the prior (e.g., Kalman and ensemble Kalman transforms, or more general techniques based on measure transport). Yet the prior-to-posterior update can also be viewed as a continuous transformation, governed by some dynamics on state space indexed by an artificial time. There are infinitely many choices of such dynamics (both deterministic or stochastic), with either finite or infinite time horizons, and any choice is associated with a transport equation encoding the particular path of probability measures taken between prior and posterior. In computational schemes used to realize these continuous-time transformations, a representation of the prior is initialized at time zero and the dynamics are simulated until a stopping time is reached, at which point the resulting probability distribution should approximate the posterior, if not realize it exactly. Computational simulation raises further questions linked to the choice of dynamics: how to compute a “step” given available information, how to choose step sizes, and how to determine stopping times for dynamics with infinite time horizons. Yet it is not well understood how the underlying choice of dynamics influences our ability to realize complex prior-to-posterior updates efficiently. In this work we elucidate connections among various frameworks which have been proposed for continuous-time Bayesian inference, and how design choices therein interact with the geometry of the probability manifold to influence performance.
Joint work with Youssef Marzouk (Massachusetts Institute of Technology).
Bayesian inverse problems in the presence of model error
Cvetkovic Nada
Eindhoven University of Technology, Netherlands - This email address is being protected from spambots. You need JavaScript enabled to view it.
In inverse problems, one often assumes a model for how the data is generated from the underlying parameter of interest. In experimental design, the goal is to choose observations to reduce uncertainty in the parameter. When the true model is unknown or expensive, an approximate model is used that has nonzero `model error' with respect to the true data-generating model. Model error can lead to biased parameter estimates. If the bias is large, uncertainty reduction around the estimate is undesirable. This raises the need for experimental design that takes model error into account.
We present a framework for model error-aware experimental design in Bayesian inverse problems. Our framework is based on Lipschitz stability results for the posterior with respect to model perturbations. We use our framework to show how one can combine experimental design with models of the model error in order to improve the results of inference.
Spectral Convergence of Symmetrized Graph Laplacian on Manifolds with Boundary
J. Wilson Peoples
The Pennsylvania State University , USA - This email address is being protected from spambots. You need JavaScript enabled to view it.
We study the spectral convergence of a symmetric, truncated graph Laplacian matrix induced by a Gaussian kernel evaluated on pairs of data, sampled from a manifold with boundary. Specifically, we deduce convergence rates for eigenpairs of this matrix to the eigensolutions of the Laplace-Beltrami operator satisfying homogeneous Dirichlet boundary conditions. We provide a detailed numerical investigation of this convergence on simple manifolds. Our method of proof combines a min-max argument over a compact and symmetric integral operator with a recent weak convergence result coming from an asymptotic expansion of a Gaussian kernel integral operator on a manifold with boundary.
Joint work with John Harlim (The Pennsylvania State University).