I think that ISPs having local monopolies would do more to drive costs up than Net Neutrality. Japan doesn’t have the same problems we have, yet they maintain net neutrality.
Let’s also look at the consequences of letting ISPs restrict content, and the way they’d change how you access the internet. We’d probably move to the “cable” model: you purchase the basic package, but have to pay extra for less visited sites. Or worse, if the ISP is the content producer and distributor, under no net neutrality they have every incentive to make their content easy to access and everyone else’s expensive to access. Pair this with local monopolies, and you get a very bad deal for the consumer.