You’ve a right to your thoughts and feelings; they’re legit. I think you have the right to say “No thanks” to people when they want you to join them in prayer.
The illustration you employ regarding nonbelievers who turn to prayer as a last resort dying more often than those with regular prayer lives under similar circumstances is too vague to be helpful to me. There are too many variables floating around in such situations to definitively hang the fault upon prayer.
People who want you to pray with them will not magically know your thinking. Tell them. Claim your right not to pray.
As for me, I pray. I expect little in the way of the miraculous, but I do feel a bit closer to God and those I join in prayer. This is helpful for me, even though I can understand that it isn’t for you.
I wish you well!
