Thank you david prompovitch!
There are software that handle your public and private keys (like GPGTools). They synchronize directly with your e-mail software.
Regarding the “reply all”, it does create problems. Sending something sensitive to multiple people in the same time is not recommended. You would have to send it individually.
You could, however have the same private key as the group you are trying to send the e-mail. This would allow you to “reply all”. I don’t think I need to explain how bad that is, though.