Find A Publication

Latest Publications

  • There is a pressing need to build an architecture that could subsume these networks undera unified framework that achieves both higher performance and less overhead. To this end, two fundamental issues are yet to be addressed. The first one is how to implement the back propagation when neuronal activations are discrete. The second one is how to remove the full-precision hidden weights in the training phase to break the bottlenecks of memory/computation consumption. To address the first issue, we present a multistep neuronal activation discretization method and a derivative approximation technique that enable the implementing the back propagation algorithm on discrete DNNs. While for the second issue, we propose a discrete state transition (DST) methodology to constrain the weights in a discrete space without saving the hidden weights. In this way, we build a unified framework that subsumes the binary or ternary networks as its special cases.More particularly, we find that when both the weights and activations become ternary values, the DNNs can be reduced to gated XNOR networks (or sparse binary networks) since only the event of non-zero weight and non-zero activation enables the control gate to start the XNOR logic operations in the original binary networks. This promises the event-driven hardware design for efficient mobile intelligence. We achieve advanced performance compared with state-of-the-art algorithms. Furthermore,the computational sparsity and the number of states in the discrete space can be flexibly modified to make it suitable for various hardware platforms. Read More
  • In this paper, we present a geometric multigrid methodology for the solution of matrix systems associated with isogeometric compatible discretizations of the generalized Stokes and Oseen problems. The methodology provably yields a pointwise divergence-free velocity field independent of the number of pre-smoothing steps, post-smoothing steps, grid levels, or cycles in a V-cycle implementation. The methodology relies upon Scwharz-style smoothers in conjunction with specially defined overlapping subdomains that respect the underlying topological structure of the generalized Stokes and Oseen problems. Numerical results in both two- and three-dimensions demonstrate the robustness of the methodology through the invariance of convergence rates with respect to grid resolution and flow parameters for the generalized Stokes problem as well as the generalized Oseen problem provided it is not advection-dominated. Read More
  • We prove a new off-diagonal asymptotic of the Bergman kernels associated to tensor powers of a positive line bundle on a compact K\"ahler manifold. We show that if the K\"ahler potential is real analytic, then the Bergman kernel accepts a complete asymptotic expansion in a neighborhood of the diagonal of shrinking size $k^{-\frac14}$. These improve the earlier results in the subject for smooth potentials, where an expansion exists in a $k^{-\frac12}$ neighborhood of the diagonal. We obtain our results by finding upper bounds of the form $C^m m!^{2}$ for the Bergman coefficients $b_m(x, \bar y)$, which is an interesting problem on its own. We find such upper bounds using the method of Berman-Berndtsson-Sj\"ostrand. We also show that sharpening these upper bounds would improve the rate of shrinking neighborhoods of the diagonal $x=y$ in our results. In the special case of metrics with local constant holomorphic sectional curvatures, we obtain off-diagonal asymptotic in a fixed (as $k \to \infty$) neighborhood of the diagonal, which recovers a result of Berman [Ber] (see Remark 3.5 of [Ber] for higher dimensions). In this case, we also find an explicit formula for the Bergman kernel mod $O(e^{-k \delta} )$. Read More
  • We study implicit regularization when optimizing an underdetermined quadratic objective over a matrix $X$ with gradient descent on a factorization of $X$. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution. Read More
  • The evidence lower bound (ELBO) appears in many algorithms for maximum likelihood estimation (MLE) with latent variables because it is a sharp lower bound of the marginal log-likelihood. For neural latent variable models, optimizing the ELBO jointly in the variational posterior and model parameters produces state-of-the-art results. Inspired by the success of the ELBO as a surrogate MLE objective, we consider the extension of the ELBO to a family of lower bounds defined by a Monte Carlo estimator of the marginal likelihood. We show that the tightness of such bounds is asymptotically related to the variance of the underlying estimator. We introduce a special case, the filtering variational objectives (FIVOs), which takes the same arguments as the ELBO and passes them through a particle filter to form a tighter bound. FIVOs can be optimized tractably with stochastic gradients, and are particularly suited to MLE in sequential latent variable models. In standard sequential generative modeling tasks we present uniform improvements over models trained with ELBO, including some whole nat-per-timestep improvements. Read More
  • In this work we have used the recent cosmic chronometers data along with the latest estimation of the local Hubble parameter value, $H_0$ at 2.4\% precision as well as the standard dark energy probes, such as the Supernovae Type Ia, baryon acoustic oscillation distance measurements, and cosmic microwave background measurements (PlanckTT $+$ lowP) to constrain a dark energy model where the dark energy is allowed to interact with the dark matter. A general equation of state of dark energy parametrized by a dimensionless parameter `$\beta$' is utilized. From our analysis, we find that the interaction is compatible with zero within the 1$\sigma$ confidence limit. We also show that the same evolution history can be reproduced by a small pressure of the dark matter. Read More