Your point that personal projects miss the element of “coding to someone else’s requirements” is well taken. Of course sometimes other people’s requirements are nonsense. I had to learn XSLT for a one-off project that I could have done more quickly in PHP, but the boss wanted the “newer” technology on HIS resume.
But in terms of self-learning, every advance I’ve made has come through a personal project, either something I wanted, or making something that existed more generic — for example a lot of PHP/MySQL tutorials start the student out using named data fields, and I would show that to a beginner to illustrate the concepts, but wouldn’t make him do it even once, so I learned PHP/MySQL by building a generic database front-end. Always thinking “now that this works, how can I genericize it”.
I’m not smart, but through personal projects I’ve learned: