You’ve tapped into one of the most caustic social habits that exist: fear of seeming like an asshole for knowing and sticking to what matters.
It’s akin to Warren Buffet’s (and Steve Job’s) “one rule for success”: the ability to say “no” to work (or, for Job’s, design features) that distracts from what you know you both need and want to achieve your goals.
And that’s where I think the true problem lies: either/both a lack of strong desire for something or awareness of what one really wants.
Without that True North it’s easy to follow tangential paths willy nilly, only to hover above “E” when you finally discover what you really desire.
And it’s at that breaking-point where we scream “NO!” and seem like the asshole we wanted to avoid being.