Good long post, thanks.
Many DL researchers, similar to ML but to a less extent, have a comfort zone in mathematics. That is true. There is even the demand by the more traditional ML folks (i.e. Vapnik) that there is not enough mathematics ( meaning, not enough theory). I prefer to look at DL as being experimental and I seek out the few experimental results that our out there to refine my intuition of how I think DL works.
I’m of course not about the criticize the high priests here about their comfort blanket. There is of course a human reason for the attachment, that I’m not about to discuss in public.
Best of luck on your research!