Stop worrying about how your servers see you, and start wondering whether your servers are making enough to make ends meet. Over time, the rest will take care of itself.
We have the tipping system to prevent awful service, not to ensure perfect service. As long as they do their job adequately, pay them. If they do an especially good job or if you take up their table for an extra long time, maybe give them a little extra (depending on circumstances, like whether or not the restaurant is busy and your presence is displacing another paying customer). Personally I have a minimum tip besides a percentage: $2 if I only get one or two items (like a drink or a drink and a side), $4 for a meal for one, $6 for a meal for two — even if the total is only $15. It all takes up their time regardless of how much the food itself is — so make sure your servers are taken care of. If they know you are taking care of them, they will take care of you.