Omnibus Tests for Treatment Effect Heterogeneity

Apoorva Lal

2022-10-23

We simulate some experimental data with effect heterogeneity, and use the testing approach in Ding, Feller, and Miratrix (2018) to test for systematic heterogeneity.

suppressPackageStartupMessages(library(causalTransportR))
set.seed(42)

# simulate RCT
treatprob <- 0.5

# workhorse
dgp <- function(n = 10000, p = 10, treat.prob = treatprob,
                # bounds of X
                Xbounds = c(-1, 1),
                # nonlinear heterogeneity
                tauF = function(x) 1 / exp(-x[3]),
                # nonlinear y0
                y0F = function(x) pmax(x[1] + x[2], 0) + sin(x[5]) * pmax(x[7], 0.5)) {
    X <- matrix(runif(n * p, Xbounds[1], Xbounds[2]), n, p)
    a <- rbinom(n, 1, treat.prob)
    # generate outcomes using supplied functions
    TAU <- apply(X, 1, tauF)
    Y0 <- apply(X, 1, y0F)
    # outcome
    y <- (a * TAU + Y0 + rnorm(n))
    list(y = y, a = a, X = X)
}

simulation with heterogeneity

with(dgp(), dfmTest(y, a, X))

## [1]  8.543073e+02 4.322510e-177

p-value \(\approx 0\) - conclude that there is systematic heterogeneity.

simulation without heterogeneity

with(dgp(tauF = function(x) 1/3), dfmTest(y, a, X))

## [1] 17.52836639  0.06346002

large p-value - conclude that there is no systematic heterogeneity.

References

Ding, P., A. Feller, and L. Miratrix. (2019): “Decomposing Treatment Effect Variation,” Journal of the American Statistical Association, 114, 304–17.