Where do all property rights come from?
If you have a gun… they come from the guy with the biggest gun.
Otherwise all property rights flow from and are enforced via government.
You have no other rights to what you call ‘your property’.
And yes, ‘the government’ not only gives them, it determines the limits. So you cannot set up a slaughter house in a residential street, or demolish property on a whim.
You may have spent your money building a house, but you have consumed community resources doing so. It is in the community’s interests to see that those resources are not squandered.
You are the owner only for the time being. The property remains for ownership/use by others potentially long after you have moved on or are gone.
A city comprises all the infrastructure services and buildings that generations past have created. We all benefit from that and have an interest in conserving it as a ‘common good’, including for generations unborn.