Methods of Causal Inference

Investigating the methodology behind the rationale

Photo by Wolfgang Hasselmann on Unsplash

In my previous post, I covered the basics of Causal Inference. It’s time to take a step further and learn about a few methods of Causal Inference:

1. Instrumental Variables (IV)

Instrumental variables (IV) are essential in causal inference, especially when dealing with potential sources of endogeneity (a statistical term that describes the correlation between an independent variable and unexplained variation in a dependent variable)or unobserved confounding variables.

An instrumental variable is a variable that is used to estimate the causal effect of an independent variable (X) on a dependent variable (Y) when there is potential endogeneity or confounding between X and Y. In other words, an instrumental variable helps us identify the causal relationship between X and Y by providing X variation independent of the factors that might confound the relationship.

Key Characteristics of an Instrumental Variable:

1. Relevance: The instrumental variable must be correlated with the independent variable (X). This ensures that it has some impact on the outcome (Y) through its association with X.
2. Exogeneity: The instrumental variable should not be directly related to the outcome variable (Y) except through its association with the independent variable (X). This ensures that the instrumental variable does not suffer from the same endogeneity issues as the independent variable.
3. Exclusivity: The instrumental variable should not be correlated with any unobserved factors that influence the outcome variable (Y) directly. This helps to isolate the causal effect of the independent variable (X) on the outcome (Y).

How Instrumental Variables Work:

1. Identifying Variation: The instrumental variable helps to identify the variation in the independent variable (X) that is not influenced by confounding factors or endogeneity issues.
2. Isolating Causal Effect: By using the instrumental variable, we can estimate the causal effect of X on Y without bias from confounding variables or endogeneity.
3. Interpreting Results: The estimated causal effect obtained through instrumental variables analysis represents the causal impact of changes in X on changes in Y, assuming the instrumental variable satisfies the key characteristics mentioned earlier.

Consider a study investigating the effect of education (X) on income (Y). However, education might be endogenous due to unobserved factors like ability. In this case, we can use an instrumental variable such as distance to the nearest college. Distance to the nearest college is likely correlated with education (as people closer to colleges are more likely to pursue higher education) but is unlikely to be directly related to income except through its effect on education. Therefore, distance to the nearest college can serve as an instrumental variable to estimate the causal effect of education on income.

Instrumental variables provide a powerful method for estimating causal relationships in situations where traditional regression techniques may lead to biased results due to endogeneity or confounding. By carefully selecting and analyzing instrumental variables, we can better understand the causal mechanisms underlying the relationships between variables in their studies.

2. Propensity Score Matching

The propensity score is the probability of receiving the treatment given a set of observed covariates. It represents the likelihood of receiving the treatment based on observed characteristics, thus serving as a summary of the confounding variables.

Propensity score matching is an advanced method in causal inference used to estimate the causal effect of a treatment, intervention, or exposure on an outcome variable in observational studies. It helps to address the issue of confounding variables(factors that influence both the treatment assignment and the outcome of interest) by creating a pseudo-randomized experiment from observational data.

For example, in a study examining the effectiveness of a new educational program on student performance, factors like socioeconomic status or prior academic achievement(covariates) could confound the relationship between the program and student outcomes.

Propensity Score Matching Process:

• In propensity score matching, individuals who have similar propensity scores but different treatment assignments are paired or matched together.
• This creates a matched sample where the distribution of observed covariates is similar between the treated and untreated groups, mimicking a randomized experiment.
• Matching can be performed using various methods such as nearest neighbor matching, kernel matching, or exact matching.

Estimating Causal Effect:

• Once the matched sample is created, experimenters can estimate the causal effect of the treatment by comparing the outcomes between the treated and untreated groups.
• Because the matched groups are balanced on observed covariates, any difference in outcomes can be attributed more confidently to the treatment itself rather than confounding variables.

Advantages and Considerations:

• Propensity score matching allows researchers to approximate the conditions of a randomized controlled trial in observational data, thereby strengthening causal inference.
• However, it requires careful selection of covariates and matching methods, and the assumption of no unobserved confounding still holds.

In summary, propensity score matching is a valuable tool in causal inference, allowing researchers to account for confounding variables and estimate causal effects in observational studies. By creating matched samples based on propensity scores, we can strengthen the validity of causal claims drawn from observational data. However, it’s important to acknowledge the assumptions and limitations of this method and to interpret results with caution.

3. Difference-in-Differences (DiD)

Difference-in-differences is a powerful method used in causal inference to estimate the causal effect of a treatment, intervention, or policy change by comparing changes in outcomes over time between a treatment group and a control group. This method is particularly useful when randomized controlled trials (RCTs) are not feasible or ethical.

Let’s dive into how DiD works.

1. Treatment Group and Control Group: In a typical difference-in-differences setup, you have two groups: a treatment group and a control group. The treatment group is exposed to the intervention or treatment of interest, while the control group is not.
2. Pre- and Post-Intervention Periods: The analysis typically involves observing outcomes for both groups before and after the intervention or treatment takes place. These periods are referred to as the pre-intervention period (before the treatment is implemented) and the post-intervention period (after the treatment is implemented).
3. Estimating the Treatment Effect: The key idea behind difference-in-differences is to compare the change in outcomes over time between the treatment group and the control group. By doing so, we can estimate the causal effect of the treatment or intervention. Specifically, we compare the difference in outcomes before and after the intervention in the treatment group with the difference in outcomes over the same periods in the control group.
4. Assumption of Parallel Trends: A crucial assumption in difference-in-differences analysis is the parallel trends assumption. This assumption states that, in the absence of treatment, the trends in outcomes over time would have been the same for both the treatment and control groups. Violations of this assumption can bias the estimated treatment effect.
5. Statistical Analysis: The treatment effect in a difference-in-differences analysis is typically estimated using regression analysis. The regression model includes indicator variables for group membership (treatment vs. control), time period (pre-intervention vs. post-intervention), and an interaction term between group membership and time period.
6. Interpreting the Results: The coefficient associated with the interaction term in the regression model represents the estimated treatment effect. A statistically significant coefficient suggests that the treatment had a causal effect on the outcome of interest.

Difference-in-differences is widely used in various fields, including economics, public health, and policy evaluation, to evaluate the impact of interventions or policy changes. It provides a robust method for estimating causal effects using observational data, but it requires careful attention to assumptions and potential sources of bias.

4. Regression Discontinuity Design (RDD)

Regression Discontinuity Design is a powerful technique used in causal inference to estimate causal effects when treatment assignment is determined by a cutoff point or threshold in a continuous variable. It’s a design often used in quasi-experimental studies where subjects are assigned to treatment or control groups based on whether they fall above or below a particular threshold.

RDD exploits the idea that units very close to the threshold are essentially randomly assigned, as they are nearly identical in all aspects except for their position concerning the cutoff point. Therefore, any difference in outcomes between those just above the threshold (treated group) and those just below it (control group) can be attributed to the treatment.

Steps in RDD:

1. Identification of Cutoff Point: RDD starts by identifying a clear cutoff point in a continuous variable that determines treatment assignment. For example, a school policy might provide extra tutoring for students scoring below a certain threshold on a standardized test.
2. Study Design and Data Collection: Researchers collect data on both sides of the cutoff point. This includes information on the outcome of interest (e.g., test scores) as well as covariates that might affect both treatment assignment and the outcome.
3. Estimation of Treatment Effect: Using statistical methods such as local linear regression or polynomial regression, researchers estimate the treatment effect by comparing outcomes just above and just below the cutoff point. The key assumption is that there are no systematic differences between the treatment and control groups except for their proximity to the cutoff.
4. Checking Assumptions: Researchers should conduct various checks to ensure the validity of the RDD design. This includes examining the continuity of the assignment variable around the cutoff, testing for manipulation of the assignment variable near the cutoff, and assessing the balance of covariates between the treatment and control groups.

Advantages of RDD:

• RDD can provide credible causal estimates in situations where randomized experiments are not feasible or ethical.
• The design of RDD is intuitive and transparent, making it easier to communicate the results to stakeholders.
• By focusing on units near the cutoff point, RDD reduces the potential for selection bias compared to traditional observational studies.

Challenges and Considerations:

• Sensitivity to Functional Form: The choice of functional form (e.g., linear vs. non-linear) in estimating the treatment effect can influence results.
• Manipulation Near the Cutoff: There may be instances where individuals manipulate their assignment variable (e.g., test scores) to gain treatment, potentially biasing the estimated treatment effect.
• External Validity: The results of an RDD may only be valid in the specific context of the cutoff point and may not generalize to other settings.

Suppose you want to evaluate the effectiveness of a program that provides financial aid to low-income students based on their family income. You could use RDD by comparing outcomes (e.g., academic performance) for students just above and just below the income threshold used to determine eligibility for the program.

Overall, Regression Discontinuity Design is a valuable tool in the causal inference toolkit, providing researchers with a rigorous method for estimating causal effects in quasi-experimental settings where treatment assignment is determined by a clear cutoff point.

If my article tickled your fancy, throw in a handful of claps, or go the extra mile and sprinkle it among your pals or work buddies!

You can also connect with me on LinkedIn.