Yet, if our world defined success in another way (e.g. buddhism) all the privilege you speak of would work against us.
That is, success as the world measures it today defines privilege.
Up to the day we started believing in the massive whopper of a story we (mostly someone else) invented to give meaning to our consciousnesses, success meant not dying each day. Back then, not dying each day was a collective and collaborative affair. A fuck-you-im-alright-jack approach was a one way ticket to death.
We needed the whopping story because we could not believe that our group effort was the only reason why we did not die that day, we knew so little so we begun to believe that some supernatural force/entity helped us out. We do not need this story anymore. We are the reason why people live another day or die before their time (other than via nature or biology).
Also, empathy is a stranger to most successful people because empathy is really understanding the lot of others by curious investigation. Successful people could not live with themselves if they really empathise with unsuccessful people — it is better for their sanity to remain in a bubble.
Success would be better measured by the simple thought that due my efforts this day/week/month/year some people did not die. If we believed this, then we might be a lot more curious about how our efforts prevented someone from dying. Do I pay enough tax and where does it go? Funding a drone strike or a housing project? Does wealth really trickle down? Do the people who I assigned authority to do enough?