Welcome to the world of the underprivileged. Congratulations on making your way out. Not everyone has the ability, capability, or the chance. You may feel that you were able to succeed on your own — and it sounds like you have the drive and fortitude to break through barriers — but in reality no one makes it without some help from someone somewhere along the line. I’m sure if you examine your success you will see there were opportunities that you were able to take advantage of that, perhaps, others may not have had. (Was there a loan you were able to get from a bank to start that business? Loans are hard to come by for some demographics.)
You may feel that your “whiteness didn’t help” you, but you would not ever know that. Unless you’ve faced discrimination or abuse based on your “whiteness” you would not have any measure to make that claim. The example of “reverse discrimination” you cite here (very real) can certainly be called an outcome of an affirmative action policy, but it was not because of your “whiteness.”
Not everyone is a “victim” but, yes, there are true “victims.”