You have a right to a view; do you have a right to pronounce it to millions of New York Times readers, however? No.
This is of course true but I don’t think the analogy applies to the case of Milo Yiannopolous or other provocative speakers.
This is because, in the newspaper analogy, the editor clearly does not agree with the white supremacists, the NASA conspiracy nuts. The editor does not deem the letters worthy of publication, and the cranks cannot cite their right to ‘free speech’ to change his or her mind.
But in the case of Milo and others, the ‘editor’ in the scenario is someone in the university. Either a person acting on behalf of the institution itself, or the chairperson of some society, faculty or group that has the right to invite external speakers.
If the university gives those students and faculty the right to invite external speakers, then it surely is a violation of free speech if people seek to prevent the ‘editorial’ or curatorial decision from being fulfilled.
The correct ‘newspaper’ analogy would be if readers of the paper picketed the printing presses of the New York Times, when they heard that the editor had accepted a particular letter for publication. If the editor capitulated in that situation, we would rightly be worried about the effect of the mob on editorial freedom.
The question must surely be: are universities de facto public spaces; or something akin to a student’s home-away-from-home? If the former conception prevails, then the free speech principles that govern actual public spaces should be adopted. But if we choose the latter conception, then of course we have every right to ban people we consider undesirable from coming into our space.