Causal AI and Causal Inference

Most machine learning systems are fundamentally correlational — they learn statistical associations between inputs and outputs from observed data. Causal AI goes further: it attempts to understand the mechanisms that generate data, enabling models that can reason about interventions, answer counterfactual questions, and generalize across environments in ways purely correlational models cannot.

Correlation vs. Causation

The distinction is fundamental to how models behave:

• Correlation: Feature $X$ is associated with outcome $Y$ in the training data.
• Causation: Changing $X$ produces a change in $Y$.

A classic example: rooster crowing is highly correlated with sunrise, but the rooster does not cause the sun to rise. A correlational model that relies on this association will fail the moment a different data-generating process is in play.

In machine learning:

• Spurious correlations arise from shared causes (confounders) or sampling biases.
• A classifier trained to detect pneumonia learned to associate the absence of a bone spur (present primarily in severe cases transferred to hospitals) with safety — the opposite of the true causal relationship.
• Image classifiers use background texture as a shortcut; NLP models rely on lexical cues that don’t generalize.

Causal reasoning allows models to distinguish robust causal features from spurious correlational ones.

The Ladder of Causation

Judea Pearl formalized three levels of causal reasoning in his Ladder of Causation:

Standard ML operates entirely at Level 1. Causal inference targets Levels 2 and 3.

Structural Causal Models (SCMs)

A Structural Causal Model formally represents a causal system as:

$$X_i := f_i(\text{Pa}(X_i), U_i)$$

where:

• $X_i$ is each variable in the system.
• $\text{Pa}(X_i)$ are the parents (direct causes) of $X_i$.
• $U_i$ is exogenous noise.
• $f_i$ is the causal mechanism.

The collection of these equations, combined with a Directed Acyclic Graph (DAG) encoding causal relationships, defines the full causal structure.

Example: Confounding

    C (Confounder)
   ↙          ↘
  X           Y

If $C$ causes both $X$ and $Y$, then $X$ and $Y$ are correlated even if there is no causal path $X \to Y$. Without adjusting for $C$, a model will learn a spurious $X \to Y$ association.

The $do$-operator

Pearl’s $do(X = x)$ operator represents an intervention — surgically setting $X$ to value $x$ regardless of its natural causes. This is distinct from conditioning ($X = x$ in the observational distribution):

$$P(Y \mid do(X = x)) \neq P(Y \mid X = x) \quad \text{when confounding exists}$$

The backdoor adjustment formula computes the interventional distribution from observational data when a valid adjustment set $Z$ (blocking all back-door paths) exists:

$$P(Y \mid do(X = x)) = \sum_z P(Y \mid X = x, Z = z) P(Z = z)$$

Causal Discovery

Causal discovery is the task of inferring causal structure (the DAG) from observational data. Key algorithms:

Constraint-Based Methods

Use conditional independence tests to determine which edges can and cannot exist:

• PC Algorithm: Starting from a fully connected graph, remove edges if the corresponding variables are conditionally independent given some set. Orient remaining edges using orientation rules.
• FCI (Fast Causal Inference): Extension of PC that handles latent confounders.

Score-Based Methods

Search for the DAG that maximizes a scoring function (e.g., BIC, BDeu):

• GES (Greedy Equivalence Search): Greedy search over equivalence classes of DAGs.
• NOTEARS: Reformulates DAG learning as a continuous optimization problem using an acyclicity constraint:

$$\min_W \mathcal{L}(W) \quad \text{subject to} \quad h(W) = \text{tr}(e^{W \circ W}) - d = 0$$

Functional Causal Models

Exploit asymmetries in the data-generating process to orient edges:

• LiNGAM (Linear Non-Gaussian Additive Models): In linear systems with non-Gaussian noise, the causal direction can be identified from the data.
• ANM (Additive Noise Models): Fit $Y = f(X) + \varepsilon$ and $X = g(Y) + \eta$ — the true causal direction typically has a better-fitting residual.

Causal Inference Methods

Given a known or assumed causal structure, these methods estimate causal effects from observational data.

Potential Outcomes Framework (Rubin)

Defines causality through potential outcomes:

• $Y_i(1)$: outcome for unit $i$ if treated ($T = 1$).
• $Y_i(0)$: outcome for unit $i$ if not treated ($T = 0$).

The individual treatment effect is $\tau_i = Y_i(1) - Y_i(0)$. Only one is ever observed (the fundamental problem of causal inference).

The Average Treatment Effect (ATE):

$$\text{ATE} = \mathbb{E}[Y(1) - Y(0)]$$

Estimation Methods

Heterogeneous Treatment Effects

Methods like Causal Forests and Meta-Learners (T-Learner, S-Learner, X-Learner) estimate how treatment effects vary across subpopulations:

$$\tau(x) = \mathbb{E}[Y(1) - Y(0) \mid X = x]$$

Causal AI and Machine Learning

Invariant Risk Minimization (IRM)

Standard ERM (Empirical Risk Minimization) minimizes average loss across the training distribution but may overfit spurious correlations. IRM trains a feature representation $\Phi$ such that the same linear classifier is optimal across all training environments:

$$\min_\Phi \sum_e \mathcal{R}^e(\Phi) \quad \text{s.t.} \quad \nabla_{w \mid w=1} \mathcal{R}^e(w \cdot \Phi) = 0 \quad \forall e$$

This encourages the model to rely on causally stable features that work across environments rather than environment-specific spurious correlations.

Causal Representation Learning

The goal is to learn disentangled representations corresponding to independent causal variables — making downstream models more robust to distribution shift:

• iVAE (Identifiable VAE): Learns latent causal factors that are identifiable given auxiliary information.
• CausalVAE: Integrates a causal graph structure into the VAE latent space.

Counterfactual Data Augmentation

Generate counterfactual training examples (what would this sample look like if only the causal variable changed?) to reduce reliance on spurious features:

• In NLP: flip sentiment words while preserving non-sentiment context.
• In vision: change image style/background while preserving semantic object identity.

Causal AI Applications

Challenges

Causal AI vs. Standard ML at a Glance

Summary

Causal AI moves machine learning from pattern matching to reasoning about the world’s underlying mechanisms. Key takeaways:

• Correlation ≠ causation — spurious associations cause models to fail under distribution shift.
• Pearl’s Ladder of Causation defines three levels: association, intervention, and counterfactual.
• SCMs and DAGs provide formal representations of causal structure.
• Causal discovery algorithms infer causal graphs from data.
• Causal inference methods estimate intervention effects from observational data.
• IRM and causal representation learning integrate causal principles into deep learning.
• Causal AI is critical for trustworthy, robust, and fair AI systems — especially in high-stakes domains like healthcare, policy, and autonomous systems.