I think we cannot afford to be not afraid. Words matter as much as the actions they precursor, exactly because the actions they precursor matter. Words might matter less, as they don’t certainly lead to actions. But some actions are so terrifying that no risk of them executed should be acceptable, therefore no words precursoring them should spread, or should be without overwhelming dispute. That’s the basis of outlawing threat of immediate and concrete harm. That’s also the drive of many great writings. Among them is The Open Society and Its Enemies by Karl Popper, from where I quoted the opening.
And that’s not even considering the power of word itself. Also related to threat, bullying has been a plague of any popular open online forum, such as Facebook, Twitter, even Wikipedia. It is also a form of existential threat, targeting online existence. The words themselves are actions, and the words themselves do enough damage to depress, or even kill. These do not contribute to public discussions either, other than their existence and consequences as warnings of their harm.
A usually used analogy of speech is market. Absolute free speech is therefore analogous to laissez-faire market. That we should let space and time to resolve it all is also the motto of laissez-faire market. Unfortunately, it has been both argued and shown that it would collapse without some control. Therefore, I strongly suspect the same for absolute free speech.