I think this is true of everyone, though to a lesser extent whatever group happens to be the majority. (While I can’t prove it, my guess is their are still stratifications in every society — from deep in South America to Aboriginal tribes in Australia.)
And it’s hard for everyone, mostly, at some point. Plenty of skinny white boys watched Schwarzenegger and Stallone and thought they didn’t measure up either.
This is not to discount your story — but more to help everyone realize that you’re not a lone — everybody faces periods of feeling “other”.