simple answer, you are right, all men will respond to your initial interest as a sexual possibility, but then most men will also respond to your next cue or the lack of it indicating the actual level of your interest. Fact is, there are some men out there, somewhere, you might just be interested in having sex with, though probably most men, the vast majority, you are not sexually interested.
As for being accused of sleeping your way to the top, you can’t control what people think. Sadly, rising in life puts a target on your back for the worst that humanity has to offer, wear it as a badge of honor, as an indication of your success, a rite of passage. Men endure it too, things like, if your daddy wasn’t a stockholder you’d a never gotten that job, etc.