SPARSITY · OBSERVED COVARIATES AND MEDIATORS

Sparsity

250 observations, 400 directly observed covariates X, and 400 directly observed mediators M. This example follows the base sparse model in Debiased Causal Mediation Analysis in Ultra-High-Dimensional Settings in the Presence of Interaction Effects, Eq. (2.3) and Section 6.1.

Download dataset (CSV) Coefficients and true effects (JSON) Generator (Python) Generator helper (Python)

Data-generating model

X ~ N(mu_X, Sigma_X), mu_X = 0.5 sqrt(50/p) * 1_p A | X ~ Bernoulli(expit((X' alpha - center) / scale)) M(a) = B_a X + U_a Y(a,m) = X' beta_a + m' gamma_a + epsilon_a M = M(A), Y = Y(A,M), a in {0,1}

Each beta and gamma vector has exactly 3 nonzero coefficients; every row of B has 3. Their supports are drawn independently and nonzero values follow Uniform(0.5, 1.5).
The propensity vector alpha has 3 nonzero values drawn from Uniform(0, 2). As in the modified research R script, center and scale are the sample mean and sample standard deviation of X' alpha. This normalization is an implementation detail beyond the paper's displayed logistic equation.
Sigma_X and Sigma_U have random orthogonal eigenvectors and eigenvalues drawn from Uniform(1, 2). U_0 and U_1 are independent N(0, Sigma_U); epsilon_0 and epsilon_1 are independent N(0, 1). All error draws are mutually independent and independent of X and treatment assignment.
B_0 differs from B_1, beta_0 from beta_1, and gamma_0 from gamma_1. This permits A × X interactions in both models and A × M interactions in the outcome model. There is no X × M interaction.
X and M are the actual variables. The dense, full-rank error covariance does not introduce latent measurement factors.

Known effects

Let v be the sample mean of the generated X for the paper's conditional simulation target, or mu_X for the population target. The expressions integrate over mediator and outcome errors.

theta00(v) = v' (beta_0 + B_0' gamma_0) theta10(v) = v' (beta_1 + B_0' gamma_1) theta11(v) = v' (beta_1 + B_1' gamma_1) NIE = theta11 - theta10 (total natural indirect effect) NDE = theta10 - theta00 (pure natural direct effect) TE = theta11 - theta00 = NIE + NDE

Target	NIE	NDE	TE
Loading the saved true effects…

Use the conditional target when checking this fixed dataset, matching the paper's convention. These are analytic truths; a fitted estimate is not expected to equal them in a single sample.

Use this dataset

Load Sparsity, confirm the supplied variable roles, and select observed covariates and mediators. Since both p and q exceed n, the toolbox routes to the sparse method and applies L1 regularization to both blocks.

The example starts at K1 = K2 = 2.75. Enabling sensitivity automatically checks the current data and fills a feasible sensitivity grid, starting from 1 to 3 in steps of 0.25. You can edit the range afterward. The main K pair is preserved. Bootstrap keeps these values fixed; the check does not guarantee that every resample is feasible.

Reproduce the draw

Save both Python files in the same folder, install NumPy if needed, and run:

python generate_sparse_example.py

The fixed seed is 2026091901. The generated JSON records the NumPy version, sparse coefficients (1-based column indices), covariance spectra, propensity normalization, CSV checksum, and both sets of true effects. The generator reconstructs the covariance rotations from the seed. It also updates the example manifest without removing other DGP families.

Source: local paper Papers/Sparsity.pdf, pp. 6 and 19–20, and the base branch of mediation_sparsity/modified/Simu_modified.R. This is an independent Python realization of that design with the research code's propensity normalization. It does not reproduce the paper's saved R random draws or its 500-replication performance tables.