Information Theory
Aug 9, 2017 · 1 min read
There is a book called Information Theory, Inference, and Learning Algorithms written by a formerly professor in Department of Physics of University of Cambridge. This book could be downloaded from http://www.inference.org.uk/mackay/itila/. Also, he taught a class, the resource is here: http://www.inference.org.uk/mackay/itprnn/. It includes videos and slides for this book.
This book helped me understand basic concept of variational method and mean-field variational which could be used as an inference approximation for probabilistic graphical models in Computer Vision. For example, Fully connected CRF used the mean-field approximation.
Mean-field variational inference
https://lingpipe-blog.com/2013/03/25/mean-field-variational-inference-made-easy/
