Panel replication: implemented corrections

Constrained synthetic control, FE inference conventions, and untreated-only imputation

Published

September 21, 2026

Scope

The initial lecture audit identified a confirmed standalone synthetic-control optimization defect, different fixed-effect inference conventions, and missing causal-panel interfaces. This implementation addresses the solver defect and adds the core FE prediction, imputation, and covariance interfaces. It does not implement the entire new-estimator roadmap.

FE imputation documentation · Fixed-effects regression documentation · Synthetic control documentation · Updated development specification

Synthetic control

The standalone estimator now solves the squared-error objective directly on the simplex with an active-set quadratic solver. Zero donor weights are permitted. The old softmax parameterization could yield nearly flat gradients at a suboptimal solution. Returning a finite parameter vector no longer suffices: convergence requires a KKT optimality check, and exhausted iteration budgets raise an error.

For Proposition 99 outcome-only SCM:

Quantity Before Corrected Independent reference
Pre-treatment RMSE 2.399913 1.656400 1.656400
Mean post-treatment gap −18.998496 −19.513630 −19.513630

The corrected solve takes 34 active-set iterations in the recorded run. summary() exposes the objective, iterations, feasibility error, normalized KKT residual and stopping reason. Tests check boundary optima, redundant donors, donor permutations, common-level shifts, single-donor inputs, failed-refit state, and insufficient iteration budgets. This is outcome-only SCM, not predictor-V SCM.

Fixed-effect regression

FixedEffectsOLS now exposes absorption tolerance and iteration limits, supports FE-only fits, compacts caller-supplied FE IDs, retains level effects, and predicts identified one/two-way FE sums. Unseen levels and combinations crossing disconnected components are rejected. Three-or-more-way level prediction remains unsupported, while slope estimation still supports it.

The default full-FE degrees-of-freedom correction is preserved. New explicit options control FE counting, observation-count correction and cluster-count correction in both summaries and Wald tests. The fatalities example verifies:

Convention Explicit settings beyond cluster IDs SE
Existing CrabbyMetrics default defaults 0.385787
fixest lecture default fe_df="non_nested" 0.357078
plm HC1 lecture calculation fe_df="none", ssc="hc1", cluster_correction=False 0.350150

All three retain the coefficient −0.639980. These are deliberate statistical conventions, not three competing numerical answers to one covariance formula.

Native FE counterfactuals

FEImputation uses the same sparse absorption machinery. It fits Y_{it}(0)=\alpha_i+\lambda_t+X_{it}'\beta only on observed positive-weight untreated cells. It accepts missing observations and treatment reversals, returns exact training/evaluation masks, and aggregates weighted observed-treated cell gaps by unit and time. It rejects unsupported FE combinations and nuisance covariates unidentified in the untreated sample.

The five available empirical surfaces are compared with the already-saved R FEct outputs below. No treated outcome enters estimation. This is point estimation, without confidence intervals or causal validation of treatment reversals.

Code 
from pathlib import Path
import os
import json
import pandas as pd
import numpy as np
import crabbymetrics as cm

root = Path(os.environ['YQX_REPLICATION_ROOT']) if 'YQX_REPLICATION_ROOT' in os.environ else next(
    p / 'replications/yqx_panel' for p in [Path.cwd(), *Path.cwd().parents]
    if (p / 'replications/yqx_panel/results/fixes').exists())
verification = json.loads((root / 'results/fixes/verification.json').read_text())
panel_results = pd.read_csv(root / 'results/fixes/panels.csv')
display(panel_results.loc[panel_results.model != 'prop99_scm',
    ['model', 'att', 'reference_att', 'max_surface_error', 'n_train', 'n_treated']])
model att reference_att max_surface_error n_train n_treated
1 gs2020_fe 0.127134 0.127134 2.397409e-10 6729.0 118.0
2 turnout_fe 1.425566 1.425566 2.129233e-08 1078.0 50.0
3 hh2019_fe 1.506416 1.506416 7.074693e-08 15135.0 2459.0
4 modern_Bischof_Wagner_2019_AJPS_full 0.082334 0.082334 7.262205e-07 374.0 160.0
5 modern_Grumbach_Sahn_2020_APSR_full 0.127749 0.127749 2.527977e-10 6729.0 118.0

Bischof’s surface agreement uses a 10^{-6} tolerance because its saved R fit has looser convergence. The other examples agree much more closely. These comparisons do not erase the initial audit’s sample/provenance differences.

Executable leakage and support check

Code 
rng = np.random.default_rng(210921)
x = rng.normal(size=(12, 8, 1))
y = rng.normal(size=(12, 1)) + np.arange(8)[None, :] / 4 + .5*x[:, :, 0]
d = np.zeros_like(y)
d[6:, 4:6] = 1  # treatment can subsequently switch off
observed = np.ones_like(d, dtype=bool)
observed[0, 0] = False
y += 2*d
m = cm.FEImputation()
m.fit(y, d, x, observed=observed)
original = m.predict()
print('ATT:', m.summary()['att'])
y[d == 1] += 100
m.fit(y, d, x, observed=observed)
np.testing.assert_allclose(m.predict(), original, equal_nan=True, atol=1e-10)
assert abs(m.summary()['att'] - 102) < 1e-8
print('No treated-outcome leakage; shifted ATT:', m.summary()['att'])
ATT: 2.0
No treated-outcome leakage; shifted ATT: 102.0

Regression protection and reproducibility

Code 
assert verification['linear']['specifications'] == 59
assert verification['linear']['coefficients'] == 102
assert verification['linear']['max_coef_error'] < 1e-8
assert verification['archive']['datasets'] == 47
assert verification['archive']['max_coef_error'] < 1e-8
print('Lecture regressions:', verification['linear'])
print('Archive regressions:', verification['archive'])
Lecture regressions: {'specifications': 59, 'coefficients': 102, 'max_coef_error': 2.277289468111121e-12}
Archive regressions: {'datasets': 47, 'max_coef_error': 2.712563298978843e-10}
  • Standard tests use original deterministic fixtures generated independently in base R. lm supplies coefficient/counterfactual references; full-design cluster score calculations supply covariance references; exhaustive simplex-face optimization supplies an independent small SCM optimum. The generation script is checked in; routine tests do not require R or a network connection.
  • Optional empirical tests replay Proposition 99, the five FEct examples, and the three fatalities covariance conventions using the ignored acquired data. They skip when the cache is unavailable rather than downloading data during CI.
  • replications/yqx_panel/verify_fixes.py reruns 59 lecture specifications and 47 archive regressions, plus SCM and imputation checks, against frozen R outputs. Post-fix results live in results/fixes/; original audit outputs are preserved. Schäfer uses the tightly converged R reference established by the initial audit.
  • Tests compare the appropriate reference specifications. Forty-seven archive regressions do not mean 47 complete replications of every estimator/inference pipeline. Two are explicitly baseline-only, and the two huge datasets remain outside the current memory budget.

Recorded implementation validation

The September 21 implementation run recorded 343 passing tests and one intermittent failure in the existing benchmark process-group cleanup test (os.killpg raises a macOS permission error after the parent exits). Both cleanup cases pass when rerun in isolation. All 24 new tests pass. No skip or change to that process runner was made, and the full suite is not labeled green. Details are in test-status.json.

The current release has separate CI and package artifacts. The historical local result above is preserved rather than presented as a current CI failure.

Remaining work

The implementation does not add causal IFEct/MCct, Sun–Abraham or group-time DID, varying-slope absorption, multiway clustering, joint event-study inference, bootstrap imputation inference, augmented/predictor-V SCM, or factorial-DID nuisance orchestration. They remain explicitly proposed in the specification. The existing absorbing-only estimators have not been silently broadened to missing or reversing treatment panels. These implementations were merged in PR #23 and are included in v0.9.1.