How is Gaussian Discriminant Analysis used (Machine Learning)
- Sketched Gaussian Model Linear Discriminant Analysis via the Randomized Kaczmarz Method(arXiv)
Author : Jocelyn T. Chi, Deanna Needell
Abstract : We present sketched linear discriminant analysis, an iterative randomized approach to binary-class Gaussian model linear discriminant analysis (LDA) for very large data. We harness a least squares formulation and mobilize the stochastic gradient descent framework. Therefore, we obtain a randomized classifier with performance that is very comparable to that of full data LDA while requiring access to only one row of the training data at a time. We present convergence guarantees for the sketched predictions on new data within a fixed number of iterations. These guarantees account for both the Gaussian modeling assumptions on the data and algorithmic randomness from the sketching procedure. Finally, we demonstrate performance with varying step-sizes and numbers of iterations. Our numerical experiments demonstrate that sketched LDA can offer a very viable alternative to full data LDA when the data may be too large for full data analysis
2. Classification of Radio Signals Using Truncated Gaussian Discriminant Analysis of Convolutional Neural Network-Derived Features(arXiv)
Author : J. B. Persons, Lauren J. Wong, W. Chris Headley, Michael C. Fowler
Abstract : To improve the utility and scalability of distributed radio frequency (RF) sensor and communication networks, reduce the need for convolutional neural network (CNN) retraining, and efficiently share learned information about signals, we examined a supervised bootstrapping approach for RF modulation classification. We show that CNN-bootstrapped features of new and existing modulation classes can be considered as mixtures of truncated Gaussian distributions, allowing for maximumlikelihood-based classification of new classes without retraining the network. In this work, the authors observed classification performance using maximum likelihood estimation of CNNbootstrapped features to be comparable to that of a CNN trained on all classes, even for those classes on which the bootstrapping CNN was not trained. This performance was achieved while reducing the number of parameters needed for new class definition from over 8 million to only 200. Furthermore, some physical features of interest, not directly labeled during training, e.g. signal-to-noise ratio (SNR), can be learned or estimated from these same CNN-derived features. Finally, we show that SNR estimation accuracy is highest when classification accuracy is lowest and therefore can be used to calibrate a confidence in the classification.
3.Joint Bayesian Gaussian discriminant analysis for speaker verification (arXiv)
Author : Yiyan Wang, Haotian Xu, Zhijian Ou
Abstract : State-of-the-art i-vector based speaker verification relies on variants of Probabilistic Linear Discriminant Analysis (PLDA) for discriminant analysis. We are mainly motivated by the recent work of the joint Bayesian (JB) method, which is originally proposed for discriminant analysis in face verification. We apply JB to speaker verification and make three contributions beyond the original JB. 1) In contrast to the EM iterations with approximated statistics in the original JB, the EM iterations with exact statistics are employed and give better performance. 2) We propose to do simultaneous diagonalization (SD) of the within-class and between-class covariance matrices to achieve efficient testing, which has broader application scope than the SVD-based efficient testing method in the original JB. 3) We scrutinize similarities and differences between various Gaussian PLDAs and JB, complementing the previous analysis of comparing JB only with Prince-Elder PLDA. Extensive experiments are conducted on NIST SRE10 core condition 5, empirically validating the superiority of JB with faster convergence rate and 9–13% EER reduction compared with state-of-the-art PLDA.
