Jonathan Zwart
Sep 5, 2018 · 1 min read

Great article and good to have — inter alia — the ideal-gas insights. Does the second law of thermodynamics also come in to the analogy?

Also: For a neural network, how does the (Shannon) information vary as a function of the number of neurons/weights? I am keen to see how this scales. Do you by any chance have a reference for this (I had trouble pulling it out of MacKay’s book)?

Bayesian angle also much appreciated!

Still getting my head round the tails of the Gaussian…

Huge thanks