Yes, Religious Influence Boosted Georgia’s Birth Rate
More Critiques, More Responses
After my last post defending the thesis that Patriarch Ilia II plausibly caused a demographically-significant baby-boom in Georgia,
replied on Twitter. He levied basically three new critiques: 1) that my use of PPP per capita was inappropriate; I should have used unemployment, 2) that the correct “treatment effect” specification was religion, not marital status (we agree parity is relevant), 3) that casually looking at the time series as I’m doing isn’t a valid approach.Let’s take these in turn. First of all, what happened to Georgia’s unemployment rate over the period in question? As a reminder, here’s Georgia’s total fertility rate:
I adopt the black line as my benchmark.
Georgia’s Baby Boom Was Not Driven By Lower Unemployment
And here’s the unemployment rate in Georgia, as well as a few other post-Soviet countries, 1995–2016.
Now, there’s a lot of research that suggests that when the unemployment rate rises, fertility should fall. But what we see in Georgia is that Georgia failed to enjoy the boom-times that were seen in Azerbaijan, Ukraine, Estonia, Latvia, Lithuania, or Russia from 2000–2007… and then had a ratchet upwards in unemployment during the recession and due to ware… and has had a slower recover than other countries as well. Here’s Georgia’s unemployment rate by sex, indexed to 2000, as well as TFR indexed to 2000.
As you can see, there’s the exact opposite correlation we might expect. As unemployment rose in Georgia, so did fertility! And when unemployment fell, fertility merely chugged along unchanged.
This is another case where
has correctly critiqued the fact that I did not discuss a relevant variable at all, but it turns out it didn’t really matter. If anything, we should say that Georgia’s unemployment rate has been relatively stable over the period, and such variation as has occurred has tended to move in the opposite direction one would expect if it were causally tied to fertility.But hold on. There is an interesting way we can look at this. Georgia, like many countries, has household surveys of employment alongside surveys of business enterprises. We can look at the rate of “formality” in employment since 2007. Maybe unemployment didn’t change, but the structure of employment changed? One indicator we may want to look at is the degree of “informality” in the labor market. That is, “formal” work tends to provide workers better benefits, better wages, more security, and more access to financial tools than informal work. Georgia’s government promotes formalization of work, especially as a means to fight tax evasion.
The black line suggests that the survey of enterprises is gradually yielding a more complete coverage of the people who answer the household survey saying they are formally employed by an enterprise. This suggests that enterprise-based work is getting more formalized. The blue line at the bottom shows that enterprise-reported work is also gradually reflecting a growing share of the population. The green line, meanwhile, shows that, over time, household-reported formal employment is gradually rising. The orange line shows that all employment is very gradually rising as a share of population.
All of these trends should enable higher fertility. But, ruh roh, look at 2007–2009! That period saw some increase in formalization, but not much, and actually saw a decline in total employment! In other words, Georgia was in the middle of a recession during that period. So while formalization might boost childbearing over the whole period, it would not explain a jump in 2007–2009 then flatness afterwards.
What groups are most likely to be impacted? Well, probably not 15–19 year olds, who aren’t likely to be facing their 3rd child, and probably not 65+ year olds, who are past reproductive age. Most likely, 30-somethings to 40s are the most plausible candidates for a religiously-driven fertility boom… and look, their unemployment rates rose fairly strongly from 2007 to 2009. Aside from the extreme ends of age, however, fertility in Georgia rose at similar rates for each age group.
All in all, there’s just no reason to think that a sudden improvement in employment conditions boosted fertility. Now,
points to this article as a source for the idea that lower unemployment may have boosted fertility. Here’s the quote:
The trouble is that that quote turns out to be wrong. Yes, GDP rose, but fertility didn’t rise during the period when GDP was rising fastest, and then it did rise when it slowed down. Unemployment, meanwhile, rose significantly during the period of higher fertility.
But this article makes a more robust claim, one
didn’t harp on as much, but which concerns me a great deal.What About Religion?
Before I published my article, I did some background research, and ran across the same paper
cites. You can find the academic version here. They use survey data you can find and download here. They construct two difference-in-difference studies: comparing general birth rates of Georgian Orthodox Christians to other faiths, and comparing higher-parity-only births across faiths. They produce the following graph:
As you can see, after 2007, there is essentially no boom in higher-parity children for any faith until 2010, when there’s a boom for both faiths, and the birth increase for any birth is very small and similar across groups. In other words, there was no significant baby boom in general in this data, and what boom did occur was the same across moms whose children could be baptized and those who could not.
They also test by “intensity” of religious belief: effects remain very small, though there’s some suggestion that higher-intensity parents had bigger gains in fertility, though it’s pretty weak. They also try to control for “peer effects,” (i.e. maybe Georgian Orthodox having more kids changes social norms, or has a widespread campaign impacting non-Georgians) by looking at different regions where the likelihood of peer effects varies: and they consistently find no effect.
It all smells pretty robust. However, there’s a problem.
Let’s start with crude birth rates. Here’s estimates of how many kids were born per year in Georgia from the official birth registry, Tsuladze’s adjustments, and my extrapolation from surveys of the Caucasus as used by the study in question:
UPDATE: After
asked me how I did this, I reviewed my spreadsheets and found an error. Fertility estimated from the survey turns out to be more like the lines below. Note the very large margins of error, large enough to swallow up any annual change. Note that average estimated fertility is far *below* what official data suggests. My argument below still stands as the survey data remains an extremely poor and extremely noisy predictor of fertility, as it has no actual fertility question, just questions about household member ages. But I will admit I made a spreadsheet error that mistakenly identified “extremely noisy and incorrectly low estimate” as “moderately noisy and incorrectly high estimate.” The correct chart is below. Note as well that the study authors do not cite which years of data they use, nor how far back they back-cast birth data from a given survey-year. As indicated below, which survey-year you select for birth rates has a phenomenally big impact on estimated fertility: yet more reason not to estimate fertility from this data.I’m sure the Georgian statistical agency will be fascinated to learn how monumentally they’ve understated fertility (UPDATE: this is now wrong; instead, Caucasus Barometer data indicates they’ve overstated fertility). This crude birth rate measure does not match population-wide data in trend or level.
Once you break it out by religion, it gets worse. Across their whole sample, they have under 400 non-Georgian women included. For the parity-information subsample, under 300. Across ten years, that means that they have less than 30 women in the non-Orthodox sample per year as their comparison. These are pretty darn small sample sizes, and from a sample where the aggregation of it simply does not match the existing population-level data. Update: This critique stands, and I also checked marital-status fertility: trends in the Caucasus data only vaguely approximate the trend in national aggregate data, with large error bands.
I looked at this survey as a data source. And I chose not to use it, because, while it’s a great survey for its intended use, it’s not so great for estimating demographics. By the way, the intended use is as a public opinion survey. It has tons of granular data on political, social, and cultural attitudes, but its demographically-related data is pretty messy, and it’s not targeted towards that topic.
In sum: agree with the study authors that studying treatment-by-religion would be nifty. But they data just isn’t there to do it well. This paper was a working paper. It was not peer-reviewed. Now, I love working papers! And they’re often wonderful! But there are sometimes rotten eggs among them, and this paper is one. The data they are using is simply insufficient for the task they’re setting it to, at least as far as I can tell from my own use of that data, and their description of their method.
In sum, while religion would be a cool way to cut the data, we just don’t have sufficient information to do it.
Well…. okay, kind of. We can get at it a different way. Different Georgian regions have different religious compositions. We can look at births by region, coding each region for its major religion. If a region has at least 40% non-Georgian Orthodox population, we’ll call it “non-Orthodox.” Otherwise, it’s Orthodox. Here’s births by region classification:
As you can see, there was a boom everywhere, but the extent of the boom varies. The biggest boom was in overwhelmingly Orthodox regions. A very-similar magnitude boom occurred in the brown line regions, those with about 15% non-Orthodox populations. The smallest regional boom was in the 40%+ non-Orthodox regions. But of course, those regions still had pluralities who were Georgian orthodox. Tbilisi, which is 94% Orthodox, had the smallest boom. However, Tbilisi had the highest pre-2007 crude birth rates of these regions; I am unable to compute regional age-adjusted fertility. Out towards the end of the period, we see non-Orthodox regions catch up with Orthodox ones, consistent with financial incentives boosting birth rates more in regions that had more relative “slack,” i.e. families that hadn’t recently pushed themselves to at-or-above previously desired fertility.
This is very rough approximation of fertility by religion; incredibly rough, nothing more than slightly suggestive. But there’s little reason a priori to choose a survey not designed for demographic use, with poorly-structured fertility questions, which yields fantastically incorrect total natality numbers, against very good aggregate data which, sadly, doesn’t allow us to compute exact religion-specific fertility. Both methods are woefully flawed, but my point is that the survey based evidence is (1) very flawed on its own and (2) not consistent with the national data aggregated by religion and region. Ergo, I’m happy chucking this study entirely overboard.
Should we even do this kind of time series study at all?
critiqued my article on twitter, suggesting that it was just glancing at the numbers, and that we couldn’t say much without more rigorous methods. He’s correct: I have not built a formal difference-in-difference model, or a synthetic control, which would both be ideally suited for this.Well, I mean, I haven’t computed the t-statistics for such a model. The marital-status graph I should essentially is a difference-in-difference model, just one for which I haven’t calculated the significant statistics. Spoiler alert: almost certainly significant. Here’s the graph to remind you:
Given the low data quality, these kinds of single-factor difference-in-difference are all we can do right now. But what we should really want is a dataset that lets us cross marital status, religion, region, age, and birth parity all at once. Ideally we’d also have a sample of women who had previously birthed 1 or 2 kids broken out by those factors, but who did not have kids post-2007.
This dataset does not exist in public form, but I actually think that Geostat does have it, if they release birth registry microdata. I’m unaware of a place to download that data. If somebody knows, please tell me!
In the meantime, it’s worthwhile to use the data we have. I created a kind of casual-synthetic-control in my look at comparable post-Soviet countries; yes I could clean it up formally, but that was basically what it was. It showed good reason to believe there was a unique baby boom in Georgia. I also created a simple difference-in-difference look at marital status. It showed a baby boom for marrieds, consistent with the patriarchal effect. I looked at income: there was no income boom. I looked at unemployment: there was no unemployment boom. I looked at religion in a very incomplete way, but at least found that the baby boom was indeed larger in regions with larger Georgian Orthodox populations.
Fun fact: Abortions also fell sharply according to surveys (though official data, which massively understate abortions vs. the surveys, do not show a decline). The observed changes in fertility by parity are also consistent with a parity-specific effect such as Patriarch Ilia II’s campaign.
But should I write all of this up formally? If I calculate my significant statistics, will these arguments be more valid? If I make it a PDF, format it for NBER, put asterisks by some numbers, and title it “Working Paper” instead of “Blog Post”, will it be more true?
No. It will not. Formal publication is worthless and we should stop doing it; instead we should all argue ferociously on twitter and in blog posts where we can rapidly revise, updated, and retract incorrect or incomplete claims. Papers create an illusion of certainty: the paper I cited above was hopelessly and catastrophically flawed (Update: The authors, in an amusing note in their conclusion, note that Orthodox and non-Orthodox populations have dissimilar pre-intervention trends, thus suggested that the whole difference-in-difference method they used may be invalid. Whoops. Though, to be honest, I think they’re too hard on themselves on that front), but fooled an extremely erudite reader in
. Why? Because it had a good brand name and was “published.” But alas, it was wrong. But because it’s a working paper at a prestigious institution, the odds it every gets substantive pushback, retraction, or revision are much, much lower.Check out my Podcast about the history of American migration.
If you like this post and want to see more research like it, I’d love for you to share it on Twitter or Facebook. Or, just as valuable for me, you can click the recommend button at the bottom of the page. Thanks!
Follow me on Twitter to keep up with what I’m writing and reading. Follow my Medium Collection at In a State of Migration if you want updates when I write new posts. And if you’re writing about migration too, feel free to submit a post to the collection!
I’m a native of Wilmore, Kentucky, a graduate of Transylvania University, and also the George Washington University’s Elliott School. My real job is as an economist at USDA’s Foreign Agricultural Service, where I analyze and forecast cotton market conditions. I’m married to a kickass Kentucky woman named Ruth.
My posts are not endorsed by and do not in any way represent the opinions of the United States government or any branch, department, agency, or division of it. My writing represents exclusively my own opinions. I did not receive any financial support or remuneration from any party for this research.
