I think this sums up some of moral/anti-truth errors in this argument. While I agree that people should be allowed to have different opinions on almost anything, there are beliefs that in and of themselves infringe on/impede the rights of others. Racism is one of those beliefs. A racist who acts on that racism through speech is infringing on one’s right to exist. History clearly shows the one-sided nature of racism against people with darker skin colors and newcomers/immigrants. The historical evidence and social science research behind skin privilege and anti-Black racism is unimpeachable. The public health and medical research behind the very real negative health effects of racism is unimpeachable.
Should free speech go so far as to protect the rights of one person to say something at the expense of the health, well-being, and life expectancy of another? I think the moral and ethical argument necessary to defend that question is much much more difficult than the argument you offer here.
You can call any public health mandate “authoritarian.” That’s exactly what the anti-vaxxers do. But they are wrong, nearly everyone realizes that fact, and the state/government often forces vaccination on persons or their children against their will. Should the state stop?
If mandatory vaccination ends, we would have another public health crisis on our hands equal to the crises from which we already suffer — gun violence and racism.