The problem with this line of argument is who gets to decide what is hate speech and what is valid debate. I would argue that calling someone a Climate Change Denier is hate speech, designed to revile those you disagree with, and therefore should be banned along with racist hate speech, if you allow censorship of public speech at all.
What happens to mature debate about the desirability of Islamic immigration if any discussion of the negative cultural practices of Muslim fundamentalists (Female genital mutilation, forced marriages, Sharia Law, Jihad, etc) is banned as Islamophobia?
In my view it is safer to protect all freedom of speech, then respond to hate speech with condemnation from the majority. Otherwise we risk throwing out the Freedom baby with the Politically Correct bathwater.