Information Theory

Li Yin
Li Yin
Aug 9, 2017 · 1 min read

There is a book called Information Theory, Inference, and Learning Algorithms written by a formerly professor in Department of Physics of University of Cambridge. This book could be downloaded from http://www.inference.org.uk/mackay/itila/. Also, he taught a class, the resource is here: http://www.inference.org.uk/mackay/itprnn/. It includes videos and slides for this book.

This book helped me understand basic concept of variational method and mean-field variational which could be used as an inference approximation for probabilistic graphical models in Computer Vision. For example, Fully connected CRF used the mean-field approximation.

Mean-field variational inference

https://lingpipe-blog.com/2013/03/25/mean-field-variational-inference-made-easy/

Li Yin

Li Yin

Computer scientist. ✍️A book in progress@https://github.com/liyin2015/Algorithms-and-Coding-Interviews.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade