I suspect you’re right, but I’m not sure we agree on how. If such systems gain traction, I personally expect to see expectations set about them needing to be transparent and controllable, including by bystanders (“you can’t Cloud around me!”). And I also expect the legal and political systems to weight in if such systems come out.
With the implication that Google Glass wearers might be taking pictures, our culture brought us the idea of the Glasshole.
A few sensationalized (or, perhaps, simply factual cherry picked) stories about the Crowdsourced ARCloud is less likely to spur a revolution in AR than it is to make AR socially unacceptable, and give us the ARsehole.
We could charge ahead anyway, and poo poo concerns, assuming society will catch up. Or we can figure out an alternative architecture or solution (for example). I do not believe this obvious solution is the only one.