Mercer kernels and integrated variance experimental design: connections between Gaussian process regression and polynomial approximation

This paper examines experimental design procedures used to develop surrogates of computational models, exploring the interplay between experimental designs and approximation algorithms. We focus on two widely used approximation approaches, Gaussian process (GP) regression and non-intrusive polynomial approximation. First, we introduce algorithms for minimizing a posterior integrated variance (IVAR) design criterion for GP regression. Our formulation treats design as a continuous optimization problem that can be solved with gradient-based methods on complex input domains, without resorting to greedy approximations. We show that minimizing IVAR in this way yields point sets with good interpolation properties, and that it enables more accurate GP regression than designs based on entropy minimization or mutual information maximization. Second, using a Mercer kernel/eigenfunction perspective on GP regression, we identify conditions under which GP regression coincides with pseudospectral polynomial approximation. Departures from these conditions can be understood as changes either to the kernel or to the experimental design itself. We then show how IVAR-optimal designs, while sacrificing discrete orthogonality of the kernel eigenfunctions, can yield lower approximation error than orthogonalizing point sets. Finally, we compare the performance of adaptive Gaussian process regression and adaptive pseudospectral approximation for several classes of target functions, identifying features that are favorable to the GP + IVAR approach.

Similar Publications

Optimal transportation provides a means of lifting distances between points on a geometric domain to distances between signals over the domain, expressed as probability distributions. On a graph, transportation problems can be used to express challenging tasks involving matching supply to demand with minimal shipment expense; in discrete language, these become minimum-cost network flow problems. Regularization typically is needed to ensure uniqueness for the linear ground distance case and to improve optimization convergence; state-of-the-art techniques employ entropic regularization on the transportation matrix. Read More

We present an error-controlled mesh refinement procedure for needle insertion simulation and apply it to the simulation of electrode implantation for deep brain stimulation, including brain shift. Our approach enables to control the error in the computation of the displacement and stress fields around the needle tip and needle shaft by suitably refining the mesh, whilst maintaining a coarser mesh in other parts of the domain. We demonstrate through academic and practical examples that our approach increases the accuracy of the displacement and stress fields around the needle without increasing the computational expense. Read More

This paper proposes the use of the Spectral method to simulate diffusive moisture transfer through porous materials, which can be strongly nonlinear and can significantly affect sensible and latent heat transfer. An alternative way for computing solutions by considering a separated representation is presented, which can be applied to both linear and nonlinear diffusive problems, considering highly moisture-dependent properties. The Spectral method is compared with the classical implicit Euler and Crank-Nicolson schemes. Read More

Skew bridges are common in highways and railway lines when non perpendicular crossings are encountered. The structural effect of skewness is an additional torsion on the bridge deck which may have a considerable effect, making its analysis and design more complex. In this paper, an analytical model following 3D beam theory is firstly derived in order to evaluate the dynamic response of skew bridges under moving loads. Read More

The scalable calculation of matrix determinants has been a bottleneck to the widespread application of many machine learning methods such as determinantal point processes, Gaussian processes, generalised Markov random fields, graph models and many others. In this work, we estimate log determinants under the framework of maximum entropy, given information in the form of moment constraints from stochastic trace estimation. The estimates demonstrate a significant improvement on state-of-the-art alternative methods, as shown on a wide variety of UFL sparse matrices. Read More

Matrix completion models are among the most common formulations of recommender systems. Recent works have showed a boost of performance of these techniques when introducing the pairwise relationships between users/items in the form of graphs, and imposing smoothness priors on these graphs. However, such techniques do not fully exploit the local stationarity structures of user/item graphs, and the number of parameters to learn is linear w. Read More

This paper introduces a cost-effective strategy to simulate the behavior of laminated plates by means of isogeometric 3D solid elements. Exploiting the high continuity of spline functions and their properties, a proper out-of-plane stress state is recovered from a coarse displacement solution using a post-processing step based on the enforcement of equilibrium in strong form. Appealing results are obtained and the method is shown to be particularly Peffective on slender composite stacks with a large number of layers. Read More

We consider a symmetric matrix, the entries of which depend linearly on some parameters. The domains of the parameters are compact real intervals. We investigate the problem of checking whether for each (or some) setting of the parameters, the matrix is positive definite (or positive semidefinite). Read More

This paper studies different signaling techniques on the continuous spectrum (CS) of nonlinear optical fiber defined by nonlinear Fourier transform. Three different signaling techniques are proposed and analyzed based on the statistics of the noise added to CS after propagation along the nonlinear optical fiber. The proposed methods are compared in terms of error performance, distance reach, and complexity. Read More

In this paper, we present a fast implementation of the Singular Value Thresholding (SVT) algorithm for matrix completion. A rank-revealing randomized singular value decomposition (R3SVD) algorithm is used to adaptively carry out partial singular value decomposition (SVD) to fast approximate the SVT operator given a desired, fixed precision. We extend the R3SVD algorithm to a recycling rank revealing randomized singular value decomposition (R4SVD) algorithm by reusing the left singular vectors obtained from the previous iteration as the approximate basis in the current iteration, where the computational cost for partial SVD at each SVT iteration is significantly reduced. Read More