Critical Analysis: How the Gaza Ministry of Health Fakes Casualty Numbers
Data do not lie, people do
I recently engaged in a discussion on this platform in which I was told to read Abraham Wyner’s March 2nd 2024 article How the Gaza Ministry of Health Fakes Casualty Numbers, published in Tablet Magazine. Dr. Wyner is a professor of statistics and data science at The Wharton School of the University of Pennsylvania.
The purpose of this article is to challenge the conclusions made in his opinion piece.
I find it extremely problematic that he begins the article by asserting a fact, only to “support” it through an irresponsible array of analytic graphs and “red flags” that don’t support the conclusion that he asserts as foregone.
I want to make clear that I am not challenging his credentials, or even statistics as an entire field of study. I am not attacking Dr. Wyner as a person. What I am challenging are the conclusions and assumptions that are made in his piece, which I assert are broadly misleading and irresponsible.
Namely, he states (emphasis added):
Here’s the problem with this data: The numbers are not real. That much is obvious to anyone who understands how naturally occurring numbers work. The casualties are not overwhelmingly women and children, and the majority may be Hamas fighters.
The data that he is referring to are casualty numbers repoted by the Gaza Ministry of Health (MoH). I contend that none of Dr. Wyner’s graphs serve as evidence to infer, much less conclude, that the Gaza Ministry of Health is lying OR that the data is fake.
Let’s first talk about sample sizes and the data that we’re playing with:
Wyner says that “we have enough” (data, that is) to arrive at the conclusion that he has already prophesied, using the data below (which is accessible from the article link):
What he is referring to is that a sample size of 30 (N) is generally considered the minimum number needed to achieve statistical significance. You can read more about that, and how this relates to the central limit theorem on your own time.
The takeaway here is that this is a guideline, a “rule of thumb,” and not a set in stone path forward. When possible, it is generally preferred to have larger sample sizes as doing so means more accurate results.
For this analysis, we see that there are 15-dates worth of data, so when comparing women to children (which we’ll do later), we get a sample size of 30.
First, Wyner creates a graph using the fourth column “total” to get this image (recreated here for purposes of this analysis):
The blue columns are showing us the total of deaths over time, and the black line — the trend line — shows us the direction and fitness of the data. By fitness I am referring to the extent the top of each bar gets to the trendline.
All this graph tells us is that there is an increase in casualties over time. This is showing the relationship between time and total deaths reported. R-squared tells us that the variable time (date) overlaps with the variable total deaths by 99.86%.
Put another way, this graph tells us that the total number of people reported killed increases for each day the war progresses. Arguably, this is what we should expect. Given that this 15-day period falls within a period of the conflict that reporting indicated was very intense, specifically Israeli bombardment, it seems reasonable to expect that this would increase at a fairly consistent rate.
Which is exactly what this particular data tells us. Nothing more, nothing less.
As long as bombs are dropped, and people are killed, the next bar in this series will always be higher than the next. This particular column of data will never be lower than the preceding ones (outside of data corrections), because it is telling us the total casualties over time. It does little to inform us of daily variation in casualties, and nothing about whether the data is fake. If the war were to stop abruptly on March 26th, 2024, the following graph would look like this:
This cannot tell us whether the Ministry of Health is fabricating this information, but it can inform us that between the beginning of December and the end of February reported deaths were higher than before and after.
Wyner says:
“…daily variation in death counts is caused by the variation in the number of strikes on residential buildings and tunnels which should result in considerable variability in the totals, but less variation in the percentage of deaths across groups.”
A couple of problems with this assessment:
- Wyner does not share with us the variation in percentage of deaths;
- Does not show daily variation across groups;
- and does not explain why the variation of the number of strikes would mean less variation in the percentage of deaths across groups;
Here is a graph that shows daily percent variance across groups:
Below is the total daily reported casualties. This specific information is easy to miss in the graph recreated above because each day is simply summing the previous total with the new daily count, so it will constantly appear to be elevating, for the most part.
Wyner suggests that this variation can only be due to willful smudging of numbers. I contend that this variation is more likely caused by the variation of tactical decisions to bomb various targets, residential areas, legitimate military targets, and that due to differences among the entire population to decide to evacuate, or where to evacuate to, that there are going to be some days where more women, kids, or men are going to be killed.
Case in point, 53 schools have been totally destroyed, and another 212 directly hit with ordinance. These facilities are going to be populated primarily by women and children, which means that on days where these kinds of targets are struck we ought to see days in which women or children are going to be overrepresented.
Not every tactical decision is going to be the same, and not every location is going to have the same percentage or proportion of men, women, and children to expect less variation among groups by day. It simply would not follow that there would be “less variation” if we expect to see a lot of variation.
Entirely absent from Wyner’s analysis is the number of explosives used in the start of this conflict. Israel reported that on October 23rd they conducted more than 400 air strikes and that they were escalating this number the next day (October 24th). By November 2nd, the Israeli Air Force had reported that they had dropped more than 6,000 bombs. The daily total deaths reported above is consistent with reporting — from Israeli forces — the fact that they escalated their bombing and then tapered off.
Arguably, one could make a stronger case for inflation of numbers by looking at satellite imagery theorizing the number of people potentially situated within the destructive spread. As of March 21st, the United Nations Satellite Center indicates that 35% of all buildings in Gaza have been destroyed or damaged. Researchers suggest that the number is closer to 70% in North Gaza alone with the total in the entire strip approaching 50% destroyed or damaged.
If satellite imaging didn’t already corroborate the scale of destruction in the most densely populated areas of Gaza Strip — which was already one of the most packed places on Earth — then there could be room to suggest that casualty data was simply fabricated. At present, though, imaging seems to show that the only places not being hit with ordinance and ground operations are places that were already abandoned from previous conflicts, farm land, and desert. As far back as December 19th, 2023, reporters were lamenting that there are no safe places in Gaza, a point that has been reiterated elsewhere for months since.
As long as there is variation in bombing decisions and geographical locations, and as long as a population of 2-million people are constantly moving around and evacuating (or not) from one place to another, there is going to be significant variation by day. This is also going to be impacted by the intensity of bombing on any given day.
When we aggregate chart four, and look at totals not broken down by demographic information, we get this:
The variance of total deaths reported by day for this 15-day window visually shows us that time is not as strongly correlated to the number of deaths, shows that the daily reported deaths are decreasing, and shows that the relationship with time is negative (as time increases, casualties decrease). The correlation — i.e., relationship — between time (days in period/conflict) and number of deaths is (-0.482 (P=0.03 [one-tailed], P=0.07 [two-tailed])) a moderate one, with 23% shared variance (R-squared).
We see a 41% decrease in reported deaths from 10/27/2023 to 11/2/2023, and a 10% total reduction from 11/4/2023 to 11/10/2023, for a total period decrease of more than 22%.
If, as Wyner suggests, the MoH is lying about their data, why would casualties decrease by nearly half inside of a week during a period of time in which almost all major news outlets were reporting on the sheer scale of Israel’s bombing campaign? The scale alone ought to indicate that there were going to be mass numbers of casualties.
Uncritically, Wyner fails to highlight the fact that three days prior to this dataset being available, Israel was bombing more than 400 targets per day, and indicating to the world that they planned to intensify their efforts.
Wyner says (referring to chart one):
This regularity is almost surely not real. One would expect quite a bit of variation day to day. In fact, the daily reported casualty count over this period averages 270 plus or minus about 15%. This is strikingly little variation.
Except, charts three, four, and five show “quite a bit of variation day to day,” (which can be explained by the number of targets bombed). Chart 1 is not going to inform us of any of these variables, and is therefore a meaningless visualization, and useless on informing anyone of nefarious reporting on the part of the MoH.
Chart six below shows us the percent difference day to day, and shows that 7 days are above the 270-per day average, and six below the 270-per day average.
When Wyner says “plus or minus about 15%,” he’s referring to the sum of percent changes in the sample, which is 13.62%.
What Wyner also doesn’t do is explain what amount of variation would be appropriate. On one day we saw 1% increase (around 3 deaths) but on another day saw 29% (around 78 more deaths, or a 26-times larger increase in the number of new reported deaths).
Additionally, the distribution of days appears fairly normal:
Creating a distribution chart for 170-days of conflict gives us similar results, but a slight right skew:
Even the quantile-quantile plot (QQ-Plot) for Wyner’s data set shows normality without skew. QQ-Plots are a visual method of analysis to assess if our distribution is normal or skewed.
A useful illustration demonstrating the absurdity of Wyner’s use of the total deaths graph vs daily deaths to suggest fake data because of a lack of daily variation can be shown using COVID-19 data from March 10th, 2020 to May 31st, 2020. Below is a graph.
Take for example graph one, total reported cases over time:
According to Wyner’s argument, this graph should inform us that everyone was making up most of the COVID-19 cases across nearly 10,000 hospitals and health departments between March and May 2020.
What if we just pick a 15-day window? Say, between April 9th and April 23rd?
This graph is nearly identical to the first one, using entirely different data from a completely different problem. What you don’t see is that there is an average of 29,455 new reported cases per day, plus or minus 12.54%. If we take Wyner’s assessment of MoH deaths reported, and apply it to COVID-19 cases, the only conclusion we can justify is that these data are fabricated because it “appears too perfect,” and because 12.5% is just too low for daily fluctuations. (Which is absurd.)
If we stretch this data beyond 15-days, to 50-days (so, April 9th to May 30th), not only does the average new cases decrease to 25,985, but the plus or minus drops to 0.65% (19-times smaller).
When we look at the daily change in new cases:
We get a completely different view of the story. Even when we see what appears to be a rhythmic pattern, it would not be appropriate to conclude — with these two graphs — that the data is fabricated. Doing so would be lazy, and intellectually dishonest, without a more systematic review of other potential reasons for this variation.
That rhythm exists, and became more pronounced during the pandemic, because cases were not reported officially during the weekends, and so Mondays to Thursdays would see a huge increase and then begin to fall. And so, we see what almost looks like a heart beat in the data. All because when information was reported is when that information was dated.
The same is happening within Gaza as death and injury reports are not being reported on the weekends anymore, as seen in chart eleven below. But this visualization is not likely caused by the MoH spending their weekend determining how to fake the data. It is more likely because health officials have to attend to an inundation of injuries in an already collapsed system that makes communication and reported extremely difficult, and so information will not be consistent. This suggests that we ought to see variation by day in ways not typically seen in otherwise normal circumstances.
An additional reason why there is likely going to be significant variation on some days is because as people are uncovered underneath the rubble, or communication is achieved from other parts of Gaza through antiquated means, these data could be for people that were deceased days, weeks, or months ago but are only now getting reported.
Wyner left out the most important piece of his narrative by making central to his argument a graph that does not speak to his point, or support it.
What happens if we look at all data from October 26th, 2023 to March 24th, 2024 and recreate the very first graph? So, instead of 15 days of conflict, we look at around 170 days?
Well, we get this:
The weird cliff that you see in the presentation of the data is due to reporting deficiencies. It captures the dates between November 11th, 2023 and December 2nd, 2023 which is when there was a temporary ceasefire and the OCHA did not provide daily updates with the exception of two on November 22nd and 23rd which captured 3,700 dead and 6,500 injured during the non-reporting that occurred for 11-days.
Below is a similar chart, showing injury data:
The bars in both of these images that go above the trendline (black lines) is consistent with timeline between the post-ceasefire escalation and shift from Northern Gaza to Southern Gaza, and further indicates a dissipating pace in the number of those killed and injured per day starting in mid-February.
Charts 11 and 12 are telling you the same story from different points of view. As daily reported deaths begin to decrease, the bars begin to fall below the trend line.
My assumption is that this is the variation that Wyner was suggesting ought to be present. But the thing is, is that if we converted the 170 days into 11 15-day periods, they’d all look pretty identical to the very first chart — because it’s too little data.
Don’t take my word for it, though, let me demonstrate it.
Here is what the 15-days preceding Wyner’s publication looks like:
You’ll notice that there is variation, except that those columns that show duplicated data (flatness) are capturing the weekends. Either way, it still achieves the same basic “near perfect” increase per day.
Here are the 15-days following Wyner’s publication:
And here is a date range in an area of the 170-day outlook above the trendline:
If it weren’t for the weekends, all 15-days would fit — with near perfection — the trendline with the exception of January 19th.
But if we only look at these tiny windows, we get a very incomplete picture. A 170-day assessment gives us the kind of ebb and flow that we’d expect to see that simply cannot get captured by using the “minimum amount of data” to achieve statistical significance.
This is why it is important to have larger sample sizes.
Looking at this even more comparatively, when we look at 14-days of conflict from October 27th to November 9th (Wyner’s dataset), we get an average of 270 people killed per day, and 602 injured. But when we compare the 14-days of reporting between March 10th and March 23rd, we see an average of 146 deaths, and 164 injured.
How is it that the number of reported daily deaths have dropped by 46% and injuries by 72% between two 15-day windows separated by more than four months? Would it not be in their best interest to continue inflating the numbers, allegedly? If the MoH were lying, anyway? What would Hamas gain by allowing such a decrease in deaths?
A 7-day and 14-day average for deaths and injuries looks like this:
If the MoH were intentionally lying, would these graphs not be inversed or simply flat?
Let’s talk a bit more about the women vs children and women vs men plots, and why this doesn’t help us — at all — assess if the MoH is making up their data. Below is a recreation of Wyner’s graph:
It’s worth pointing out that in Wyner’s article it shows P=0.6437 on the graph that I recreated above. This means that the F-value, R (correlation coefficient), and R-Squared are not significant.
Effectively, it suggests that there is a 64.3% chance that the correlation between women and children is due to chance. This value alone means we ought to wright this entire analysis off and look for other variables that may explain the relationship between child and adult female deaths.
Like, perhaps, the number of bombs and artillery shells that are dropped each day, when bombs are generally dropped, geographic location, targeting parameters, reporting barriers, etc.
Wyner suggests (emphasis added):
…on the days with many women casualties there should be large numbers of children casualties, and on the days when just a few women are reported to have been killed, just a few children should be reported.
This assumption rests on the idea that kids will only ever be next to their mother when they perish, which isn’t always going to be the case. While this is a logical assessment for very young children, like infants, it does not follow for older youth.
It also ignores how carpet bombing in a very condensed strip of land can result in a lot of orphans. According to UNICEF, as of early February 2024, more than 17,000 children were unaccompanied or separated from their parents. So, if there are orphaned children meandering about in the streets without adults, there is a non-zero chance that there will be instances in which ordinances kill young people and very few adults.
By around the same time as UNICEF’s report, an additional report from Save the Children indicated that more than 610,000 children were consolidated to only 20% of Gaza’s total geographical spread.
Using COVID-19 as an example, there was only 37.3% shared variance (R-squared with a correlation of 0.61, P=0.034) between adult female deaths and child deaths, and 39.6% shared variance (P=0.028) between adult male deaths and child deaths from COVID-19. This suggests that there are other larger drivers for child deaths from COVID. On the other hand, adult female and adult male deaths are strongly correlated, with 99.8% shared variance (P=3.37E-15 | <0.000).
To this end, Wyner’s point would be right in the context of COVID-19, in which we have fairly clear patterns of transmission pathways. But armed conflict may not function in the same way that virus transmission does. Just because a child’s adult guardian catches a bomb doesn’t mean that a child will, and even then, once that ordinance has detonated there is no chance that the child is at risk of an ordinance developing. The odds are, to some extent, random, and dependent on unknowable or unobtainable variables.
I am further baffled that Wyner does not delve into population statistics for Gaza to benchmark his assumptions. Per NPR, 47.3% of Gaza’s population are under 18 years of age. This leaves 52.7% of the population as adults, with a 26.35% split between adult men and women. This suggests that for every woman killed we should see 1.79 children face the same fate.
Casualty data from the 15-day window Wyner uses shows us that 1.2 children died for each adult woman, which is below expectation if we are assuming that civilian men, women, and children are at equal odds of getting obliterated. We have no reason to believe that there would be equal odds, especially if Israel is doing everything they can to minimize civilian casualties. Arguably, equal odds would suggest indiscriminate bombing.
Additionally, the data shows that 39.3% of casualties are children (16.9% below population parameters) and 32.5% are adult women (19% higher than population parameters). This is above the often reported figure of 70% for women and children that Wyner contests is fake, but below what we would expect at 73.65% assuming all ages/demographics are equally likely to be killed. This difference is less than 3%.
But there’s another way to test if this ratio is due to propaganda and fallacious number smudging.
Data — freely accessible from the Gaza Ministry of Health — shows that there have been 13,143 people confirmed killed (confirmed with the use of personal identification numbers).
Demographic breakdown shows that 36% have been young people 0–1, 26% have been women 18+, and 37% have been men 18+.
A histogram of all ages:
The variation by age further dissuades us from being able to conclude that the MoH is making up information, but more to the point we see that young people in their 20’s are more likely to die which suggests that they are either combatants or able bodied people trying to facilitate recovery efforts.
One may question the fact that the updated spreadsheet only shows 14,000+ confirmed killed, when reports are showing more than 35,000+ in the news. It is fair to be suspicious, and seek a broader and more robust assessment of that figure. A figure that all but a few countries in the world trust, by the way, including the United States Department of Defense and State Department.
This discrepancy though can be due to a lot of reasons, but is most likely driven by the collapse of the healthcare system in Gaza. Comparing what is reported to what is confirmable can help us determine if there is inflation in ratios, numbers, or rates. While data quality has diminished, and reporting is likely through less official means than before, it remains inappropriate to conclude that the data is fabricated.
Moving on, the correlation between women and men does not say anything about whether the MoH is lying. It just shows a relationship that could be spurious, or driven by other things. Wyner doesn’t explore any possible confounding variables, rather, he just claims that the MoH is fabricating data because there is a negative correlation between adult male and female deaths.
The plot at issue is this:
This negative correlation is statistically significant (regression analysis), and tells us that as one value goes up the the other value goes down. (Positive correlations tell us that both variables move in the same direction.)
Wyner argues that this should be opposite (a positive correlation), and fluid (like the COVID-19 example I gave above). Never-mind the fact that there are observable “ebbs and flows” based on daily counts rather than the sum total, this plot tells us that as more men die fewer women die.
An argument could be made that we should expect to see a negative correlation between adult men and women if adult men make up the primary fighting force and those that drive recovery efforts.
As more adult men die in combat, and as air strikes are more accurately targeted we should see more adult men die per day than adult women.
It is confounding that Wyner would look at a correlation — that he suggests with no explanation should be oppoisite — and then conclude that this must be caused by fake data. The responsible thing to do would be to look for actual confounding variables, which could be fabricated data. But he does not do this.
Again, unlike a transmissible virus that is more likely to kill adults than children, it does not follow that targeted bombing campaigns against valid military targets should result in a positively correlated ratio of civilian woman and male combatants. Especially if women do not participate in military activities, recovery efforts, and are evacuated to refugee camps.
Finally, I want to address a final series of points that Wyner makes, which I will go through point by point.
The Gaza Health Ministry has consistently claimed that about 70% of the casualties are women or children. This total is far higher than the numbers reported in earlier conflicts with Israel.
It is important for readers to understand that previous conflicts can serve as a barometer but that things are going to differ between each one. Wyner acknowledges this when he rightfully states “this war is wholly unlike its predecessors in scale or scope,” which makes his allegations even more confusing.
As stated above, 73% of Gaza’s population are adult women and children. A reporting of 70% could mean that Hamas is engaging in unsupported propaganda. Alternatively, it could mean that Israeli bombing is indiscriminate and far more pronounced than in the past. It could mean that there are other variables we should try to identify and tease out.
Given that more than 35% to 50% of structures have been destroyed or damaged, it logically follows that this would result in more civilian casualties than previous conflicts between Israel and Palestine, especially since Gaza is one of the most densely populated places on Earth.
Reporters from across the world have indicated that the current conflict between Hamas and Israel is more significant, bloody, and destructive than any other conflict between the two. Ergo, some assumptions may not be as strong as before.
Confirmed casualties are presently 9 times higher than the 2008 war, 5 times higher than the 2014 conflict, and 50 times higher than the scuffle in 2021. Reported deaths are between 24 and 700 times higher.
Another red flag, raised by Salo Aizenberg and written about extensively, is that if 70% of the casualties are women and children and 25% of the population is adult male, then either Israel is not successfully eliminating Hamas fighters or adult male casualty counts are extremely low. This by itself strongly suggests that the numbers are at a minimum grossly inaccurate and quite probably outright faked.
Again, adult male casualties based on available confirmed data indicates a casualty rate of 37%. In a conflict in which almost all Hamas fighters are likely to be adult males, we should see them overrepresented compared to population parameters.
Even in the 15-day assessment used by Wyner the ratio is 33/39/28 (women/children/men). This indicates that adult men and women are overrepresented since they make up 61% of the casualties in this date range when they only make up 52% of the population.
Finally, on Feb. 15, Hamas admitted to losing 6,000 of its fighters, which represents more than 20% of the total number of casualties reported.
On Feb. 15, the total number of people reported as killed was 28,775 (and another 68,552 injured). This does make up 20% of those reported killed.
How this is a red flag is unknown and unexplained.
This suggests that 62% of adult male casualties are combatants. Which, we ought to expect as combatants should be the most likely to perish in armed conflict, especially if we’re looking only at male casualties.
It is valid to question how it is that non-combatant men are underrepresented, but there are reasonable, rational, and logical variables that likely exist. It is lazy and dishonest not to do so, and then conclude that it must just be fake.
One variable could be that able bodied men are away from military targets as they are providing relief efforts and aid to areas and people that have already experienced bombardment, or searching for food or the dead. All being reasonable assumptions given the looming famine that the entire Gaza population faces.
That said, the imbalance between adult female and child casualties appears to have occurred in the 2014 conflict. Reportedly, there were 2,251 casualties in that conflict with 1,462 civilians killed, of which 95% were adult women and children. This meant that only 5% of the reported civilian deaths were non-combatant adult men. The current conflict gives us a ratio of twice that at 11% of potential civilian adult men being among those killed.
Others have examined the detrimental and disproportionate effects on women and children in armed conflict that fit even in the context of the current war between Israel and Hamas.
Most likely, the Hamas ministry settled on a daily total arbitrarily. We know this because the daily totals increase too consistently to be real. Then they assigned about 70% of the total to be women and children, splitting that amount randomly from day to day. Then they in-filled the number of men as set by the predetermined total. This explains all the data observed.
Wyner’s claims are not “most likely.” He provides no evidence beyond hearsay, assumptions, “red flags,” and “circumstantial evidence” to support this claim. It borders on theoretical, and is lightyears away from anything we can call reality.
What is evident is that absent any legitimate evidence, the only option for Wyner was to attempt credibility was to con the common folk with graphs and an appeal to authority using a very tiny window of the conflict that predated his article by more than four months. What happened in his 15-day window is not statistically different from any other random 15-day window over the entire period.
As an expert, he knows this, and is banking on the likelihood that you do not.
One thing that is evident is that as hospitals continue to be attacked, raided, and bombed, the capacity of health officials and doctors to report deaths accurately becomes increasingly more difficult, if not impossible. This then puts the burden of these measures on the media through reporting which becomes prone to error.
Israel knows this.
Saying that we should see variation, not showing how the variation appears or would be observed, and then labeling variation as intentionally manifested randomness — without evidence — is not just questionable and wrong, it is intellectually dishonest.
It is wholly inappropriate for an individual of Wyner’s pedigree (PhD in statistics and data science) to so uncritically and lazily conclude that the numbers are just figments of some institution’s imagination.
Such an accusation does not harm Hamas as the basis of this rests on accusing health professionals — people that went to medical school, swore an oath, and that manage the dead — of lying. It is an accusation without any foundation of fact. The entire premise is absurd.
The overwhelming historical information and preponderance of evidence suggests that prior to the deterioration of the entire health system in Gaza in January 2024 that the reporting by the Gaza Ministry of Health was valid, based on protocols similar to systems in other countries, and validated by numerous independent sources, including the Untied States and Israel.
Wyner’s 15-day window of data falls squarely within a timeframe that was considered as accurate.
As the situation in Gaza has grown more complex, more destructive, and more disjointed, the capacity of health officials to accurately report deaths and injuries has become extremely difficult and the data quality has diminished. This is factual.
It is worthwhile to question — and assess — to what extent the data is quality and that the reporting is accurate.
Wyner’s piece does not do this, and is an example by which appeals to expertise are used to churn out propaganda that is meant to dissuade individuals from trusting what is being reported.
We also ought to critique Israel’s continued assault on the healthcare system in Gaza as it is that system that is responsible for reporting this information. Destroying that system is not only a war crime, but it means that Palestinian’s in Gaza can simply be written off as liars as verifiable data becomes impossible to capture and report. This is one way in which Palestinian’s are dehumanized, and it perpetuates a hostile cacophony of phony ideas that only serve to ensure that the entire population of Gaza gets punished.
