Dynamics of Quadratic convergence part3(Machine Learning 2024)
- A Globally Convergent Policy Gradient Method for Linear Quadratic Gaussian (LQG) Control
Authors: Tomonori Sadamoto, Fumiya Nakamata
Abstract: We present a model-based globally convergent policy gradient method (PGM) for linear quadratic Gaussian (LQG) control. Firstly, we establish equivalence between optimizing dynamic output feedback controllers and designing a static feedback gain for a system represented by a finite-length input-output history (IOH). This IOH-based approach allows us to explore LQG controllers within a parameter space defined by IOH gains. Secondly, by considering a control law comprising the IOH gain and a sufficiently small random perturbation, we show that the cost function, evaluated through the control law over IOH gains, is gradient-dominant and locally smooth, ensuring the global linear convergence of the PGM. Numerical simulations show that the dynamic controller learned by the proposed PGM is almost same as the LQG optimal controller, indicating promising results even in a reduced-order controller design.
2. Convergence Rates of Quadratic Transform and WMMSE Methods
Authors: Kaiming Shen, Ziping Zhao, Yannan Chen, Zepeng Zhang, Hei Victor Cheng
Abstract: Fractional programming (FP) plays an important role in information science because of the Cramer-Rao bound,the Fisher information, and the signal-to-interference-plus-noise ratio (SINR). A state-of-the-art method called the quadratic transform has been extensively used to address the FP problems. This work aims to accelerate the quadratic transform-based iterative optimization via gradient projection and extrapolation. The main contributions of this work are three-fold. First, we relate the quadratic transform to the gradient projection, thereby eliminating the matrix inverse operation from the iterative optimization; our result generalizes the weighted sum-of-rates (WSR) maximization algorithm in [1] to a wide range of FP problems. Second, based on this connection to gradient projection, we incorporate Nesterov’s extrapolation strategy [2] into the quadratic transform so as to accelerate the convergence of the iterative optimization. Third, from a minorization-maximization (MM) point of view, we examine the convergence rates of the conventional quadratic transform methods — which include the weighted minimum mean square error (WMMSE) algorithm as a special case — and the proposed accelerated ones. Moreover, we illustrate the practical use of the accelerated quadratic transform in two popular application cases of future wireless networks: (i) integrated sensing and communication (ISAC) and (ii) massive multiple-input multiple-output (MIMO).
3. On the quadratic convergence of Newton’s method for Mean Field Games with non-separable Hamiltonian
Authors: Fabio Camilli, Qing Tang
Abstract: We analyze asymptotic convergence properties of Newton’s method for a class of evolutive Mean Field Games systems with non-separable Hamiltonian arising in mean field type models with congestion. We prove the well posedness of the Mean Field Game system with non-separable Hamiltonian and of the linear system giving the Newton iterations. Then, by forward induction and assuming that the initial guess is sufficiently close to the solution of problem, we show a quadratic rate of convergence for the approximation of the Mean Field Game system by Newton’s method. We also consider the case of a nonlocal coupling, but with separable Hamiltonian, and we show a similar rate of convergence
