How Energy Based Models work part3(Machine Learning)
- On finite time singularities in scalar field dark energy models based in the RS-II Braneworld(arXiv)
Author : Oem Trivedi, Maxim Khlopov
Abstract : The quest of deciphering the true nature of dark energy has proven to be one of the most exciting in recent times in cosmology. Various ideas have been put forward in this regard besides the usual cosmological constant approach, ranging from scalar field based models like Quintessence and Phantom dark energy to various modified gravity approaches as well. A very interesting idea then is to consider scalar field dark energy models in quantum gravitationally corrected cosmologies with the RS-II Braneworld being one of the most well known in this regard. So in this work, we consider RS-II Braneworld based scalar field dark energy models and try to look out for the existence of finite time singularities in these regimes both through a dynamical system perspective, for which we employ the Goriely-Hyde singularity analysis method, and a physical perspective. Our approach is general in the sense that it is not limited to any particular class of potentials or for any constrained parameter region for the brane tension and is valid for both Quintessence and phantom dark energy regimes. We firstly show through Goriely-Hyde procedure that finite time singularities can exist in these models for a limited set of initial conditions and that this result would hold irrespective of any consideration given to the swampland dS conjecture. We then discuss the physical nature of the singularities that can occur in this regime, where we use a well motivated ansatz for the Hubble parameter and show that these models of dark energy can allow for weak singularities like those of Type III and Type IV and can also allow for strong singularities like the Big Rip (Type I)
2. Sampling from Discrete Energy-Based Models with Quality/Efficiency Trade-off(arXiv)
Author : Bryan Eikema, Germán Kruszewski, Hady Elsahar, Marc Dymetman
Abstract : Energy-Based Models (EBMs) allow for extremely flexible specifications of probability distributions. However, they do not provide a mechanism for obtaining exact samples from these distributions. Monte Carlo techniques can aid us in obtaining samples if some proposal distribution that we can easily sample from is available. For instance, rejection sampling can provide exact samples but is often difficult or impossible to apply due to the need to find a proposal distribution that upper-bounds the target distribution everywhere. Approximate Markov chain Monte Carlo sampling techniques like Metropolis-Hastings are usually easier to design, exploiting a local proposal distribution that performs local edits on an evolving sample. However, these techniques can be inefficient due to the local nature of the proposal distribution and do not provide an estimate of the quality of their samples. In this work, we propose a new approximate sampling technique, Quasi Rejection Sampling (QRS), that allows for a trade-off between sampling efficiency and sampling quality, while providing explicit convergence bounds and diagnostics. QRS capitalizes on the availability of high-quality global proposal distributions obtained from deep learning models. We demonstrate the effectiveness of QRS sampling for discrete EBMs over text for the tasks of controlled text generation with distributional constraints and paraphrase generation. We show that we can sample from such EBMs with arbitrary precision at the cost of sampling efficiency
3. Perturb-and-max-product: Sampling and learning in discrete energy-based models(arXiv)
Author : Miguel Lazaro-Gredilla, Antoine Dedieu, Dileep George
Abstract : Perturb-and-MAP offers an elegant approach to approximately sample from a energy-based model (EBM) by computing the maximum-a-posteriori (MAP) configuration of a perturbed version of the model. Sampling in turn enables learning. However, this line of research has been hindered by the general intractability of the MAP computation. Very few works venture outside tractable models, and when they do, they use linear programming approaches, which as we will show, have several limitations. In this work we present perturb-and-max-product (PMP), a parallel and scalable mechanism for sampling and learning in discrete EBMs. Models can be arbitrary as long as they are built using tractable factors. We show that (a) for Ising models, PMP is orders of magnitude faster than Gibbs and Gibbs-with-Gradients (GWG) at learning and generating samples of similar or better quality; (b) PMP is able to learn and sample from RBMs; © in a large, entangled graphical model in which Gibbs and GWG fail to mix, PMP succeeds.
