I’ve heard this more than once, that even a thousand good deeds of a person go down the drain for one bad word. This is the sad reality of humans in general. Simply said no one truly and wholeheartedly appreciates whatever good you do, even if its a personal favour for someone. Everyone, however is more than willing to rake up all your faults at the strike of the bell, so to speak, given the chance.
Simply said, it is a part of human nature. So is listening to others. In my opinion what you do is perfectly okay in your own eyes, if that is the case then it becomes irrelevant what anyone else thinks. Because they may even misinterpret your good as bad. The fact remains that no matter what you do, the world will conspire to taint your image.