Pr(non-VAX|ICU) ≠ Pr(ICU|non-VAX)

I already showed in another post that science is human work. This speaks for itself in the purest sense of the word, because machines don’t do science (yet). We do. With all the consequences that this entails, because everything that is human is inherently frail and fallible. Unfortunately, we seem to forget this every now and then, putting way too much emphasis on what science shows us. As if science is pure, flawless, all-encompassing.

I’m not going to repeat what I wrote in my previous post, but what I’m going to do is dig deeper into ‘dependence’. Conditional probability. To explain this, I would again like to use Bayes’ theorem which goes like this:

1. P(A|B) is the dependent probability: the probability of A if B is true. This is also known as the posterior probability.
2. P(B|A) is also a dependent probability: the probability of B if A is true. This is also called the likelihood of A given a fixed B because P(B|A) = L(A|B).
3. P(A) and P(B) are the probabilities of A and B, respectively. These are marginal probabilities which means that the probability is not dependent. This is called the prior probability.

Again, the keyword is dependency. A depends on B and B depends on A. Only, that dependency is NOT the same. That’s what I want to talk about in this post — the counter-intuitive in Bayes’ theorem when you’re just looking at the data without including the bigger world. The joke is that Bayes’ theorem is a mathematical representation of how we as humans actually learn. Namely iteratively, but also quite demarcated from the bigger picture. The statement “ignorance is bliss” is thus the precursor of cognitive dissonance.

Let me make it concrete by specifically focusing on the relationship between Covid-19, ICU admission and vaccination status. There is a lot to do here lately.

In Dutch, the number of positive tests through time.

In Dutch, the number of ICU admission across time.

In Dutch, the level of vaccination — 88.5% for first shots, and 84.9% for second shots.

What I now hear more and more is that it is the group of unvaccinated people who fill the ICU beds. That data is freely available, albeit highly aggregated and lacking a host of key variables at a more granular level. Nevertheless, we can count on it. See the RIVM chart below.

In Dutch, the rows show age, and the columns show non-vax, partial-vax, and complete-vax (2 shots).

Other available sources are tables such as this one from the CWZ, but for the sake of convenience I will calculate with the RIVM table. Precisely because it emphasizes that it is the unvaccinated who end up in the ICU.

To get off to a good start, we must immediately state that Pr(VAX|IC) is not Pr(IC|VAX). These are two positions on opposite sides of the same coin, but are NOT interchangeable:

1. The chance that someone will enter the ICU given vaccination status — Pr(A|B).
2. The vaccination status given that someone enters the ICU — Pr(B|A).

What often goes wrong is that the above tables are seen as ‘proof’ for Pr(A|B), but in fact it is Pr(B|A). In fact, it is even Pr(B,C|A) because it is the chance that someone has been vaccinated, and has reached a certain age, that he or she will end up in the ICU. That’s where we get our information from. This is therefore not equal to Pr(A|B,C) — the chance that someone will end up in the ICU given his age and vaccination status.

The difference, I will already reveal, lies in the fact that the tables do not look at the “prior probability” of even ending up at the ICU. Which is about non-existent. If 3000 Covid-19 patients come to the ICU per year, that is 3000 / 17.5 million people. That is the basic chance of ending up in the ICU. On top of that comes age and vaccination status and you actually want to include many more variables. Unfortunately, RIVM does not have any raw data available, and so we have to make due with limited information. As mentioned, this is not a problem for this exercise.

To calculate further I need additional information that is not available in the table so I used the CBS and RIVM websites to create the context. Monk’s work.

p ("12-49")           = (8406602  / 17475415)         = 0.4
p ("50-69")           = (4691222  / 17475415)         = 0.27  
p (">70")             = (2424970  / 17475415)         = 0.14
p (">12")             = (15522794 / 17475415)         = 0.89


p (VAX | "12-49")     = (6118208  / 8406602)          = 0.73
p (nVAX| "12-49")     = (2288394  / 8406602)          = 0.27
p (VAX | "50-69")     = (4145529  / 4691222)          = 0.88
p (nVAX| "50-69")     = (545693   / 4691222)          = 0.12
p (VAX | ">70")       = (2222492  / 2424970)          = 0.92
p (nVAX| ">70")       = (202478   / 2424970)          = 0.08

Then, off to the RIVM table data.

Pr("12-49"|IC)           = (453      / 1777)          = 0.26
Pr("50-69"|IC)           = (872      / 1777)          = 0.49
Pr(">70"  |IC)           = (452      / 1777)          = 0.25

Pr(VAX|IC,  ">70")       = (178      / 452)           = 0.3938 
Pr(nVAX|IC, ">70")       = ((265+9)  / 452)           = 0.6062
Pr(VAX|IC,  "12-49")     = (11       / 453)           = 0.0243
Pr(nVAX|IC, "12-49")     = ((431+11) / 453)           = 0.9757
Pr(VAX|IC,  "50-69")     = (169      / 872)           = 0.1938 
Pr(nVAX|IC, "50-69")     = ((674+29) / 872)           = 0.8062

The last six rows are especially interesting. If we explain the ratio of probabilities for vaccinated or unvaccinated (whereby I have considered partially vaccinated as unvaccinated) then it looks like this:

• “>70” = 0.6062/0.3938 = 1.53
• “12–49” = 0.9757/0.0243 = 40.11
• “50–69” = 0.8062/0.1938 = 4.16

Impressive. The chance of finding an unvaccinated “12–49” year old in the ICU is 40 times greater than a vaccinated one. Only, these are marginal ratios and these ratios are only interesting if you want to run as fast as you can across the ICU to find an unvaccinated 35-year-old. Not so interesting if you want to know what the chances are that an unvaccinated 35-year-old will end up in the ICU. In fact, I don’t even know what the odds are of finding a 35-year-old unvaccinated because the data is pretty broadly classified. I also do not know whether I will find a man or woman and how long this person has been there. I don’t really know anything about these people, except vaccination status and a broad estimate of age.

However, lets not spoil the fun. Let’s calculate the probability of actually entering the ICU given the vaccination status and the age category.

Described below, in code, for the age “>70” the probability of entering the ICU.

Pr(ICU|">70")       = Pr(">70"|IC)*Pr(IC)/Pr(">70")
                    = (0.25* 0.0001)/ 0.14=0.0001785714

Pr(ICU|VAX,">70")   = Pr(VAX|ICU,">70")*Pr(ICU|">70")/Pr(VAX|">70") 
                    = (0.3938*0.0001785714)/0.92=0.0007276739 

Pr(ICU|nVAX,">70")  = Pr(nVAX|ICU,">70")*Pr(ICU|">70")/Pr(nAX|">70")
                    = (0.6062*0.0001785714)/0.08=0.001288175

Lets compare probabilities, for VAX and non-VAX. We can see that the probability of ending up in the ICU, at age 70 or more and based on non-VAX or VAX is: 1:776 and 1:1374, respectively.

Pr(nVAX|ICU,">70") ≠  Pr(ICU|nVAX,">70")
0.6062             ≠  0.001288175        IT IS 470x MORE
Pr(VAX|ICU,">70")  ≠  Pr(ICU|VAX,">70")
0.3938             ≠  0.0007276739       IT IS 541x MORE

And then, lets look at compare chances of ending up in the ICU when 70 years or older.

Pr(ICU|nVAX,">70") ≠ Pr(ICU|VAX,">70")
0.001288175        ≠ 0.0007276739       IT IS 1.77x MORE

Together, the above proves that a simple table containing ICU patients by VAX status and age does not offer the true probability of ending up in the ICU. IN fact, the true probability is around 500 times less. These are numbers worth communicating! Nevertheless, the chance of going to the ICU being VAXED is 1.77 times less.

Let’s finish the exercise for the other groups.

Pr(ICU|"12-49")     = Pr("12-49"|ICU)*Pr(ICU)/Pr("12-49")
                    = (0.26*0.0001)/0.48=0.00005

Pr(ICU|VAX,"12-49") = Pr(VAX|ICU,"12-49")*Pr(ICU|"12-49")/Pr(VAX|"12-49")
                    = (0.0243*0.00005)/0.73=0.0000016 
Pr(ICU|nVAX,"12-49")= Pr(nVAX|ICU,"12-49")*Pr(ICU|"12-49")/Pr(nVAX|"12-49")
                    = (0.9757*0.00005)/0.27=0.0001806852

Now we have the posterior probabilities of getting into the ICU based on VAX status and age, lets compare the numbers.

Pr(nVAX|ICU,"12-49") ≠  Pr(ICU|nVAX,"12-49")
0.9757               ≠  0.0001806852        IT IS 5400x MORE
Pr(VAX|ICU,"12-49")  ≠  Pr(ICU|VAX,"12-49")
0.0243               ≠  0.0000016           IT IS 15187x MORE

As you can see, the table is once again not very useful and severely overestimates the true probability of ending up at the ICU, which is 1:5000 for non-Vax people and 1:625000 for VAX people.

Pr(ICU|nVAX,"12-49") ≠ Pr(ICU|VAX,"12-49")
0.0001806852         ≠  0.0000016            IT IS 112x MORE

Comparing the posterior probabilities shows that for the 12–49 age-group, the non-VAX group has a 112x higher probability. However, the probability of ending at the ICU, given the limited data we have now, is about 1:5000.

And if we finish the exercise for the latest group, the 50–69 age-category, we end up with these numbers.

Pr(ICU|"50-69")     = Pr("50-69"|ICU)*Pr(ICU)/Pr("50-69")
                    = (0.49*0.0001806852)/0.27 = 0.0001814815  

Pr(ICU|VAX,"50-69") = Pr(VAX|ICU,"50-69")*Pr(ICU|"50-69")/Pr(VAX|"50-69")
                    = (0.1938*0.0001814815)/0.88 = 0.00003996718

Pr(ICU|nVAX,"50-69")= Pr(nVAX|ICU,"50-69")*Pr(ICU|"50-69")/Pr(nVAX|"50-69")
                    = (0.8062*0.0001814815)/0.12 = 0.001219253

And if we compare them, we see the same happening here as what we have seen above and that is that probabilities based on a data-table DOES NOT produce the true probability of a person ending up at the ICU. Once again, it is severely overestimated.

Pr(nVAX|ICU,"50-69") ≠  Pr(ICU|nVAX,"50-69")
0.8062               ≠  0.001219253 IT IS 661x MORE
Pr(VAX|ICU,"50-69")  ≠  Pr(ICU|VAX,"50-69")
0.1938               ≠  0.00003996718 IT IS 4848x MORE

And when we do a final comparison we see that the probability of ending up at the ICU is 30x higher for the non-VAX group. The probability for a non-VAX 50–69 year old ending up at the ICU is about 1:1000. Considering what we have seen in the ICU data for the Netherlands so far, that would strike me as a severe overestimation. Which means we need more contextual data, which we are not getting.

Pr(ICU|nVAX,"50-69") ≠ Pr(ICU|VAX,"50-69")
0.001219253          ≠ 0.00003996718        IT IS 30x MORE

To sum it all up, we can make the following statements:

1. Pr(nonVAX|ICU) ≠ Pr(ICU|nonVAX). The latter are the true posterior probabilities given the data at hand
2. Non-vaccinated people do worse, however, their baseline numbers are not that bad. Also, the 12–49 groups makes little sense with such high numbers in the non-Vax group. By age alone, they should have a higher baseline number.
3. These calculations are not nearly enough to get a good sense of what is going on. Probabilities need context and having age and VAX status does not cut it.
4. And I have not even included any adverse events coming from the vaccines in the first place.

Comments and corrections to calculations are more than welcome. Science is human work, human work is fallible and I am absolutely human when it comes to my calculations. Transparency is paramount to me.

Hence, I want to offer my sincere thanks to Rene Smit who showed me an overlooked error on the conditional base probabilities of Pr((n)VAX|IC, Age). The results you see now are the corrected ones. If there are still errors, please let me know!

Mlearning.ai Submission Suggestions