There’s a difference between saying that the american right “still hates gays” and saying that trans people are “vastly disproportionately involved in sex crimes”, as Milo did recently. The first is a criticism of the behaviour of a group based on political ideology. Trans people on the other hand don’t necessarily subscribe to any particular ideology and being trans doesn’t make them any more likely to behave a particular way (aside from transitioning). Likewise for the other vulnerable groups Milo insults.

Another difference is that accusations of racism serve a legitimate purpose in at least some contexts — presumably you would not object to groups like the KKK being called racist. If I’m called a racist I can react by trying to understand the other person’s perspective and understanding of racism. I want to see the end of racism, so I would want to try to learn from it.

How though am I supposed to react to a statement like Milo’s, as a trans person? We are disproportionately the victims of sex crimes, so his statement is technically not wrong. But the meaning and the effect are clear, and the effect is that it contributes to an environment which regularly interferes with me just trying to live my life.