Hmmm…. saying that humans have good priors is just the same as saying we have good models of reality.
Well, unfortunately, as I alluded to in my response on the previous article, we don’t have good priors (or models) for F3. We got human intuition, plus mathematical theory that we’ve built up over the generations.
Unfortunately, the mathematical models barely exists for F3. We our at best having to work at a very coarse level (at the level of entire models). We don’t know of models that can generate new kinds of layers. We don’t know of models that can generate new kinds of learners. We don’t know of models that can generate new kinds of objective functions. We pretty much don’t know squat!
Perhaps it is not a mathematical model but rather a machine model. If this is the case, we may not understand the building blocks of F3.