This is a huge part of the problem. A lot of people have figured out that by putting people in a tough situation in the right way works to their advantage.
Think about it, in this instance, the guy will most likely not face any repercussions and he can keep on fishing.
Think about how often this situation plays out. How often have you ever been in a situation or heard of someone in a situation where someone asked for something that they really shouldn’t have. Yet the person being asked is the one likely to deal with the ramifications, not the person asking.
In this particular scenario, this guy is a creep. But that’s not how it will likely play out. You’ll get labeled a b@#$ for declining or a whore for accepting. What kind of screwed up choice is that?
I wish I had something to offer. I wish I knew what to tell you to help you deal with this. Unfortunately I don’t. I am contemplating these issues myself (not the same as your scenario) and I have yet to figure out a reasonable solution. The only thing is to cast these people into the light. Unfortunately, as you stated, you will most likely catch some fallout from these actions. Sigh.