The dress up happens with both genders not just one. However I agree with your last paragraph completely. The way people perceive is based on the previous encounters. If they are used to seeing a specific gender, specific dress code, specific style and grooming. They won’t appreciate people who deliver similar content in their own way. Also its more to do with how others in the room have been dressed which results in such feedback.
Rest of the writeup is just brilliant.