The standard answer to this problem (at least in Australia) is third-party data linkage and de-identification before handover to researchers for analysis. However, among the many problems with this standard answer are:

1. The time and cost associated with consultation with multiple individual data custodians or (if the data have been centralised as you suggest with some sort of “sharing agreement”) data sponsors;
2. The time and cost of the data linkage process itself (which has to be found for the third-party linkers, who won’t do it for free or authorship, unlike my data management friend down the corridor at the research institute);
3. The data sharing agreement the researcher has to sign to finally get the data, which usually stipulate single specific purpose use of the data (stopping innovation and `blue-sky’ hypothesis generating analysis) and (of very great concern to me )increasingly specify pre-publication `first-look’ and veto powers for either the data custodians or sponsors.

I agree that not only a re-think about the balance among ethical imperatives is required, but first that the successful and respectful use of large public data sets answering important questions for the public good (including, but not limited to, public health questions) needs to be publicised and celebrated by researchers in the public domain — repeatedly. Only then will the Orwellian spectre of Big Brother attached to the acquisition, storage and analysis of public data diminish … I hope!