Venue-based Tracing: Some Insights From New Zealand
Europe has been stuck with a digital approach to contact tracing tied to Bluetooth. New Zealand has been using both Bluetooth and QR-codes for a while, and have had a few cases recently. The first of those was a dedicated app user, so she offered a real-life test case in a setting where we might learn quite a lot about the functioning of the app. Given that Switzerland cannot get itself out of the Bluetooth/SwissCovid mindset, I did a Freedom of Information request to the NZ health authority to learn more.
I attach the exchange first as screenshots (it reads better, I feel), then comment a bit, then paste the email bodies as text.
Screenshots
My initial request
Their first response
My follow up questions
Their second response
Comments
In short, we get that the index case did not infect anyone. There were two notifications through Bluetooth. I infer at least one of the two Bluetooth notifications was most likely not traced to a location (and was thus most likely not a known contact). The other one was traced through Bluetooth and QR-codes. There were 700 QR-code notifications.
These 700 notifications could be considered overwhelming since they correspond to just one case. I see a few factors to take into account when evaluating this number: 1/ This corresponds to “normal life”: New Zealand was not under lockdown at the time 2/ They don’t mean that everyone should go get a test. The recommendation might be to pay heightened attention, and if you get tested positive absolutely inform the tracers of the place this notification was tied to. 3/ There will be clustering effects tied to notifications, which could lead to stronger logic: “if you receive two QR-notifications in a short enough time span, definitely do a test” (easier to implement those, as QR-notifications carry a more definite real-world significance than Bluetooth traces).
Here is a way to think about this number. Imagine everyone would be using the QR-code app as diligently as that case, in a city of 100k inhabitants. Imagine the QR-code notifications are useful if at any one time you have a 50/50 chance on average of having received notification over the past 14 days. Then that corresponds to 100000/700/14/2~ 5 cases per day.
A few other noteworthy elements:
- New Zealand actively communicates where cases have been (although they don’t communicate all the places);
- because of the previous point, QR-code can already be useful if only one side uses the app;
- New Zealanders have no appetite for COVID risk (i.e. education is key, as previously indicated when re-analyzing SwissCovid data)
- Because of the situation in New Zealand (very low incidence), this only compares the two methods in a forward tracing scenario. Only location-based tracing can realistically contribute to backward tracing — or even hybrid tracing — which is expected to have a much higher epidemiological impact, due to specific characteristics of this virus.
Update (July 1st and 2nd 2021)
I asked if there was anything new on June 22nd 2021. There was nothing new right away, but shortly after there was (see also text below).
The two links mentioned are here and here.
Further follow up, after I asked for clarification on the exact cascade:
Copy/pastes
My initial request
Dear data team at the New Zealand health ministry,
First of all congratulations to you and your ministry on your stellar response to COVID.
I would like to know if you have any available statistics on either the number of notifications, number of tests and/or number of cases found that are attributable to the contact tracing app, as well as a breakdown of each of those due to the QR-code or the Bluetooth channels.
I am particularly interested in comparing between the utility of each of those two channels in any way possible.
My motivation, living in Switzerland, is that we currently have an app deployed here that exclusively uses Bluetooth, and little traction for a QR-code based system. The theory says such a system would be much more effective than Bluetooth. Due to your low number of cases, I expect you might only have anecdotal evidence but this would still give tangible data to share here.
Thank you very much for your help,
Paul-Olivier Dehaye
Their first response
Hi Paul-Olivier,
We have taken great inspiration from the Swiss reporting of Exposure Notifications statistics, and they were useful for us in modelling how it would work in New Zealand, but I’m afraid haven’t implemented any kind of similar reporting!
We publish some data daily on https://www.health.govt.nz/our-work/diseases-and-conditions/covid-19-novel-coronavirus/covid-19-data-and-statistics/covid-19-nz-covid-tracer-app-data but not yet information about alerts.
The first usage of Bluetooth tracing (our implementation of EN) since adding it to the app in December was in late January. The first community case since adding Bluetooth tracing uploaded keys, from which two people were notified. This was in line with our modelling — our configuration of EN is very narrow. One of those people had also received a location alert (from the QR code based system). Thankfully the person who uploaded keys didn’t pass on COVID to anyone, including their very close contacts, so we can’t say that we’ve found any new cases due to EN.
For comparison, about 700 people received location notifications from locations related to that case, who had been moving around the community for a bit more than a week. It’s worth noting that we have no restrictions on movement currently. We don’t collect statistics about how many of those were tested, but as mentioned nobody else caught COVID from the index case in this outbreak.
There has been another outbreak more recently but no Bluetooth tracing keys uploaded — the more recent cases have not been users of the app. Another person also uploaded keys more recently but the bundle was empty, so no alerts were received.
About 1400 people have received location notifications in this more recent outbreak, which is currently at seven cases.
Location notifications work well for New Zealand because outbreaks are, and the population has an extremely low appetite for COVID risk. In a community where there is ongoing community transmission, the experience might be designed differently.
The biggest benefit I see of location notifications compared to Bluetooth notifications is that they can be sent out even when the index case did not use the app. They also give more information and context that people can use to make decisions.
I suggest that you have a look at the experience of the UK, which uses both systems. My understanding is that they use the Bluetooth system much more heavily, and also don’t require the display of QR codes in as many types of business. They also have an automated system for issuing these notifications, which we don’t do — there is always a contact tracer in the loop making the call on whether to alert for locations.
I’ll note that displaying QR codes is mandated for all businesses here, and the app had very limited usage before that happened.
All the best,
XXXXX
Ministry of Health
My follow up questions
Thank you very much for your detailed response, it is very appreciated!
I am a bit puzzled by the results out of England. It is very unclear to me what effect the QR code is having there, and the recent study that came out on the correlation of epidemiological trajectory and app prevalence didn’t discuss — as far as I understood — the confounding factor of the QR-code, but attributed all benefits to the Bluetooth tracing. I also remember seeing a number of QR-checkins in the 100s of 1000s, which seemed very low to me. I suppose app usage matters, and it will itself depend heavily on how much health authorities understood of the utility and how much they communicated that understanding to the general public.
(Please consider privacy impacts before attempting to answer, going for many questions now:)
a/ Were the two people who were notified via Bluetooth also identified via manual tracing?
b/ Would they have been given a bit more time?
c/ Were you able to pinpoint when/where the Bluetooth beacons were exchanged? (you would not have needed, but it might be good to think about doing this: if your Bluetooth contact had turned out positive, you could have expected to have other cases infected there but not using Bluetooth, so recovering the location/time would have been helpful).
d/ Did I understand right that the person who was notified via Bluetooth and QR was notified via QR-code first?
e/ Do you know if any of those notified people were sufficiently close/familiar contacts that these two persons were even notified directly by the index case first, themself?
f/ Why was the second uploader’s bundle empty?
g/ Which version of the API are you using?
h/ What are the Bluetooth settings you are using? (I am very familiar with how the Bluetooth setup works)
i/ Is your modeling publicly available?
I fully agree with you on where the main difference is: with QR-code you only need one user to have the app in order to already reach significant utility. Another big difference is the overdispersion of SARS-Cov-2 (it spreads in 10% of 1-to-many bursts, 70% of 1-to-0 duds and 20% of 1–1 limps). You won’t have experienced this given your current levels, but this means that in case there is someone who suddenly pops up infected “out of nowhere” (which is currently the case all over Europe), a search for the source of infection will be biased towards big events where many people would have been infected, “reaping” more benefits. QR-codes will precisely play into this strength. It is true that transposing your setup over to Europe would likely overwhelm the system. However your statement “They also give more information and context that people can use to make decisions” is equally applicable to the apps themselves. One possibility for instance would be to only notify people based on QR-codes upon *two* cases attached to the same QR-code. This would already blunt the impact of large spreading events, a very significant contribution in itself.
Thank you once again for your response so far, I am hoping I haven’t pulled too much on your generosity by asking more.
Paul
Their second response
Hi Paul — no worries at all, it’s good to nerd out a bit on this.
a/ Were the two people who were notified via Bluetooth also identified via manual tracing?
One was, one wasn’t.
b/ Would they have been given a bit more time?
Very unlikely.
c/ Were you able to pinpoint when/where the Bluetooth beacons were exchanged? (you would not have needed, but it might be good to think about doing this: if your Bluetooth contact had turned out positive, you could have expected to have other cases infected there but not using Bluetooth, so recovering the location/time would have been helpful).
For one of them yes; for the other one not to my knowledge. The case was described as an ‘assiduous’ user of the app (see story here), so contact tracers had a very good idea of where they had been very quickly. This is the other benefit of the QR code scanning — those who use the app regularly can recall their movements quickly from one data source, although QR code posters need to be placed basically everywhere for this to be possible.
d/ Did I understand right that the person who was notified via Bluetooth and QR was notified via QR-code first?
Yes, although that’s not necessarily the order it’s intended to go in. There were some issues with the Bluetooth upload, whereas the location alert process is a pretty well-oiled machine at this point.
e/ Do you know if any of those notified people were sufficiently close/familiar contacts that these two persons were even notified directly by the index case first, themself?
I’ll give this one a pass for privacy reasons, but you can infer some of the answers from the other ones.
f/ Why was the second uploader’s bundle empty?
Unknown — possibly they’d just turned it on that day.
g/ Which version of the API are you using?
V1. Trying to switch to V2 at the moment and having some problems!
h/ What are the Bluetooth settings you are using? (I am very familiar with how the Bluetooth setup works)
I’ll get back to you on this one — it’s the same as the Irish one but with a 0.5 in it somewhere.
i/ Is your modeling publicly available?
We’ve got some documentation in the docs repo about Bluetooth more generally, but I don’t think we have anything specific to NZ. Have asked some relevant people about whether we can release it.
A research institute here has done some of their own modelling on this — that’s available here. A news organisation also drew on this for some other modelling — that’s available here.
Your points about overdispersion and the different ways that the system could be applied in higher-prevalence areas are interesting ones. The problem with only alerting people who’ve been at a place where multiple other cases have been identified is that it might be too late to get any benefit from isolating them at that point though. Could be worth doing some modelling on it though — maybe you still benefit from identifying asymptomatic spreaders.
Maybe there could be a risk threshold that you set for yourself as a user of the app — perhaps you accept the risk that you were at the supermarket at the same time as someone with COVID and you don’t want to get a push notification about it, but you want to know if three people who were at the supermarket at the same time as you have contracted COVID. Maybe at one place you were wearing a mask and only stayed for a minute, but at another place you sat inside and had a loud conversation with strangers for hours. We’ve had some conversations around these kinds of risk appetite features, but appetite for COVID risk here is effectively zero for everyone.
Happy to answer any further questions you have.
XXXXX
Ministry of Health
Update (July 1st and 2nd 2021)
Hi Paul,
Not long after I sent you this we did activate both notification systems after a visitor from Australia went to a number of venues here while potentially infectious with the delta variant of COVID-19.
578 devices received location alerts, and 4 devices received Bluetooth tracing alerts, between last Wednesday and this Monday. We have identified a few thousand people who were potentially exposed at one of the places that this person went to, through a combination of app notifications, media coverage, and word of mouth, all of whom have been advised to isolate and get tested (you can see the more detailed advice on our website).
We’ve updated our EN configuration to be a little broader than it was in recognition of the new variants, but it’s still (intentionally) casting a relatively narrow net.
It’s been about a week and a half and there haven’t been any positive tests from the identified contacts, and wastewater testing is coming out negative, so we’re hopeful that there wasn’t any transmission at the venues identified.
Cheers
XXXXXXX
Ministry of Health
Further follow up, after I asked for clarification on the exact cascade:
Hi Paul,
Notifications are triggered by a clinical team at the Ministry of Health. You can see technical details about this here.
If someone tests positive, they are invited to upload their digital diary to contact tracers — you can see the process of this here. This provides contact tracers with the location codes that allow for notifications to be sent, but these location codes can also be queried separately. This means that it isn’t necessary for a case to have been an app user in order to send notifications, which is a large part of why the location alert system has been used much more than the Bluetooth tracing system.
Whether to send a notification for a particular location is a clinical decision, not automated at all, and effectively separate from the diary upload process. People may scan in to a place where no one unknown was present, so it may be possible to contact everyone exposed at that location without needing to send a notification through the app — but even if that record isn’t used to send a notification, it’s still useful for people to have the record and share it with contact tracers, so that they can jog their memory about where they’ve been and who they were near.
Cheers
XXXXXXX
Ministry of Health
