This is all nice and easy. The real problem is when your answer to the question “What’s the worst thing that might happen if you do this or that?” is really negative… Yes, in most of the situations those consequences are minor and you can easily try your bold new move without fearing them. But!
But life is not that simple. You are not always 20 years old and can take those risks easily. Older you get more your decisions are reflected on your family or your children. So that sweet egotistical approach (which is fine when you are 18–25 years old) is not always the best choice. And some “dream limitation” doesn’t necessarily make your life less happier.
Happiness is not only what I want but what is the best for me and my close family. And often family (kids, spouse, parents) should come first. It might affect some aspects of your egotistical “personal” happiness but that’s ok. Happy life, truly deep and happy existence is not about you and your feelings only. It’s about you and those people that are inseparable part of your life.