The point of the market telling polluters to clean up after themselves is to include the true cost of production within the transaction, so that taxpayers do not have to pay for the cleanup. The market failing does not imply that we should do away with markets. On the contrary, it defends the market. When we include the cleanup price that the government would be cleaning up in the price of the good, individuals are still free to determine if the good is still valuable to them even if it is a higher price — but then we remove the necessity for taxpayers to pay for cleanup.
Of course, companies would not be happy with this because a higher price means a higher equilibrium price and therefore lower number of goods sold depending on elasticity — and lower returns to them. We also have to consider equity with this, as well. But the major question still remains — what is the cost to society of polluters? Does pollution violate the assumptions of perfect competition?
The thought experiment regarding medium and the government is useful to have, but I believe that is why economists get such a bad rap. So many thought experiments but we have to look at the reality — air and water pollution are expensive to society, and the gains of pollution go to those emitters, not to society. So does it matter whether or not we use pragmatarian markets for taxes or not? The government is still subsidizing pollution if we do not include the cost of pollution in the cost of production— pragmatarianism or not.