Causal Inference in Observational Data

Introduction

Machine learning typically excels at correlation discovery, but real-world decision-making requires understanding causation. When we can’t run randomized experiments, we must infer causal relationships from observational data—a challenging but essential capability in healthcare, economics, policy, and many other domains.

The Fundamental Problem

Correlation vs. Causation

Correlation: X and Y vary together (∃ relationship)
Causation: X causes Y (manipulating X changes Y)
Confounding: Z causes both X and Y, creating spurious correlation

Why This Matters

Policy Decisions: Does program X reduce poverty?
Medical Treatment: Does drug A cure disease B?
Business Strategy: Does marketing spend increase sales?
Fair ML: Which features truly cause outcomes vs. proxies for protected attributes?

Causal Graphs and DAGs

Directed Acyclic Graphs (DAGs)

     Confounding Variable
            Z
           / \
          v   v
         X --> Y

In this DAG:

Z confounds the X→Y relationship
Observing Z closes the “backdoor path” X←Z→Y
After controlling for Z, we can infer X→Y

Graphical Model Components

Nodes: Variables of interest
Edges: Causal relationships (direction matters)
Colliders: Mediating variables (bad to condition on)
Forks: Common causes (good to condition on)
Chains: Mediating variables (shouldn’t condition on for direct effect)

Fundamental Methods

Randomized Controlled Trials (RCTs)

The gold standard but often impossible/unethical:

Pros: Eliminates confounding through randomization
Cons: Expensive, limited scope, ethical constraints
Trade-off: High internal validity, lower external validity

Adjustment for Confounders

Method: Condition on known confounders to isolate causal effect

E[Y|do(X=x)] ≈ Σ_z P(Z=z) * E[Y|X=x, Z=z]

Challenges:

Hidden/unmeasured confounders invalidate results
Over-conditioning on colliders introduces bias
High-dimensional confounding hard to handle

Stratification and Matching

Exact Matching: Find untreated units identical to treated units
Propensity Score Matching: Match on probability of treatment
Covariate Balancing: Ensure balance on key variables
Limitations: Can’t match on unobserved variables, small sample issues

Instrumental Variables (IV)

For situations with unmeasured confounding:

Z → X → Y
    ↑    ↑
    └────┘ (unobserved confounding)

If Z affects X only through causing X (exclusion restriction):

Z is an instrument
IV estimator: Effect of X on Y = Effect(Z→Y) / Effect(Z→X)

Challenges:

Finding valid instruments is hard
Weak instruments cause biased estimates
Exclusion restriction is often unverifiable

Advanced Techniques

Causal Discovery

Learn the causal graph structure from data:

Constraint-Based Methods: PC algorithm, FCI
Score-Based Methods: Hill climbing on graph scores
Functional Causal Models: Leverage asymmetry in causal direction
Limitations: Identifiability issues, orientation ambiguity

Doubly Robust Estimation

Combines outcome regression and propensity score weighting:

Consistent if either model is correctly specified
More robust than single approach
Better variance properties
Reduced sensitivity to model misspecification

Targeted Maximum Likelihood Estimation (TMLE)

Debiased machine learning approach:

Initial outcome model prediction
Propensity score estimation
Targeted update for causal effect
Reduces bias while maintaining efficiency

Causal Forests

Random forests adapted for causal inference:

Estimate heterogeneous treatment effects
Who benefits most from treatment?
Identify causal decision rules
Uncertainty quantification built-in

Addressing Selection Bias

Missing Not At Random (MNAR)

Data missingness depends on unobserved variables:

Requires strong assumptions for handling
Sensitivity analysis for assumption robustness
Inverse probability weighting adaptations

Sample Selection Bias

Study population differs systematically from target:

Truncation in dependent variable (censoring)
Heckman correction for two-stage modeling
Reweighting based on selection mechanism

Attrition Bias

Differential loss to follow-up:

Inverse probability of attrition weighting
Multiple imputation for missing outcomes
Worst-case scenario analysis

Heterogeneous Treatment Effects

Beyond Average Effects

Not everyone responds to treatment identically:

CATE (Conditional Average Treatment Effect): Effect given X
GATE (Group Average Treatment Effect): Effect for subgroups
ITE (Individual Treatment Effect): Effect for specific person

Discovery Methods

Causal Trees: Recursive partitioning for treatment effect subgroups
S-Learner: Single model with treatment indicator
T-Learner: Separate treatment and control models
X-Learner: Two-stage metalearner approach
Causal Forests: Ensemble extension with inference

Applications

Personalized Medicine: Which patients benefit from treatment?
Precision Policy: Which populations benefit from programs?
Targeted Marketing: Which customers respond to offers?

Practical Challenges

Unobserved Confounding

Variables you don’t measure but affect outcomes
Often unavoidable in observational studies
Requires sensitivity analysis and domain knowledge
May require qualitative evidence to inform structure

Small Sample Sizes

Observational studies often have limited power
High-dimensional confounding exacerbates this
Need careful model selection and regularization
Bootstrap and permutation tests for inference

Model Misspecification

True causal model may be complex
Linear assumptions often violated
Non-parametric methods have their own assumptions
Cross-validation helps but doesn’t eliminate risk

Causal Discovery Limitations

Multiple DAGs consistent with same data
Cannot distinguish from observational data alone
Requires domain expertise to resolve ambiguity
Assumptions rarely fully justified

Best Practices

1. Start with Theory

Document causal assumptions explicitly
Create DAGs before analyzing data
Identify potential confounders a priori
Consider alternative causal structures

2. Sensitivity Analysis

Test robustness to violations of assumptions
Bound bias under different scenarios
Report when conclusions become fragile
Transparent about limitations

3. Multiple Methods

Use several techniques and compare
Consistency across methods increases confidence
Different methods make different assumptions
Disagreement suggests deeper investigation needed

4. Domain Expertise

Collaborate with subject matter experts
Validate against prior knowledge
Test implications in new contexts
Iterative refinement based on feedback

5. Transparent Reporting

Clear statement of causal assumptions
Limitations and violations acknowledged
Alternative interpretations discussed
Confidence intervals on estimates

Tools and Software

R Packages

causalml: Multiple causal inference methods
grf: Causal forests and generalized random forests
DoubleML: Doubly robust and debiased ML
dagitty: DAG visualization and d-separation
randomForest, glmnet: Base ML methods

Python Libraries

EconML: Microsoft’s causal ML library
DoWhy: IBM’s causal inference framework
Causal Impact: Google’s approach to intervention analysis
scikit-learn: Base ML tools
NetworkX: Graph operations

Specialized Software

Lavaan: Structural equation modeling
MPlus: Latent variable models
STATA: Econometric causal methods
Julia Packages: Performance-critical applications

Case Studies

Healthcare

Effect of treatment on patient outcomes
Controlled for many potential confounders
Leveraged instrumental variables for unmeasured confounding
Heterogeneous effects by patient subgroup

Economics

Impact of educational programs on earnings
Causal forest to identify who benefits most
Instrumental variables for self-selection bias
Policy implications for resource allocation

Marketing

Effect of advertising on sales
Controlled for seasonality and market conditions
TMLE for unbiased effect estimation
Heterogeneous effects by customer segment

Research Frontiers

Integration with Machine Learning

Deep learning for causal effect estimation
Combining symbolic and statistical causality
Causal explanations for black-box models
Reinforcement learning with causal understanding

Automated Causal Discovery

Learning DAGs from data more reliably
Combining observational and experimental data
Multi-environment causal learning
Time series and temporal causality

Causal Fairness

Fairness as causal concept
Unfair discrimination detection
Individual vs. group fairness trade-offs
Intersection with privacy and transparency

Resources

Pearl, Glymour, & Jewell: “Causal Inference: The Mixtape” (free online book)
Imbens & Angrist: Econometric approaches to causal inference
Rotnitzky, van der Laan: Targeted learning theory
Conference: International Conference on Learning and Reasoning about Causality
Journals: Journal of Causal Inference, Epidemiology

Conclusion

Causal inference from observational data is complex but essential for impact-driven applications. Success requires combining rigorous statistical methods, domain expertise, and transparent reporting of assumptions and limitations. No single technique solves all problems—practitioners must thoughtfully select and validate approaches appropriate for their context.