I think that you confuse “being extremely secure” as a requirement to have “inapropiate banter” with “having inapropiate banter” as a sing that people fell “extremely secure”. To put an example, is like saying that “all men have two eyes” implies that “all creatures that have two eyes are men”. See?
To make it clear, safe work places allow for inapropiate banter, inapropiate banter doesn’t make safe work places. For you is automatic which, great for you. Not so for other people. Respect and trust are earned not default.
It is widely know that many women fell unconfortable when ther receive sexual comments. In fact, some men find this quite “funny” (from their point of view). They would say such things until the point it is clear that the woman is really unconfortable but not to the point they could be called out. and when the womas ask to stop they would say “it just a joke” in order to avoid consecuences. Thay might even say that “they are sorry if their comments have offended someone” impliying “thin skin”, meaning that it is the woman’s fault for not putting out with what he knew fron the begining that would make her fell unconfortable.
To be bold, they are making someone fell unconfortable on purpose to have fun at her expense, but don’t want to be considered assholes. They might even “like” the woman and be “friendly” after that. But that wont chnge the fact that they didn’t care that she fell unconfortable in order for them to have fun. And they’ll keep doing it even if the woman explain that she doesn’t like it.
Funny thing? The don’t like it when they are the butt of the “joke”. Benefit of doubt is nice, but after someone tells you “I don’t like those comments” there is no doubt, if you keep doing it, you do it knowing that the person doesn’t like it. So no “Damocles sword”, just respect and common sense.