Skip to content

feat(etwfe): unsupported_period_action opt-out for comparison-support period filtering (M-147) - #821

Open
igerber wants to merge 1 commit into
mainfrom
codex/eval-api-7ef42b41
Open

feat(etwfe): unsupported_period_action opt-out for comparison-support period filtering (M-147)#821
igerber wants to merge 1 commit into
mainfrom
codex/eval-api-7ef42b41

Conversation

@igerber

@igerber igerber commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • Adds WooldridgeDiD(unsupported_period_action="drop" | "error"), the opt-out for
    per-period comparison-support filtering that ledger row M-125 listed as NOT
    INCLUDED and TODO.md tracked. An unsupported period is one lacking the
    required comparison support: no positive-weight eligible comparison observation
    (never-treated on the never_treated + OLS branch; never-treated or
    not-yet-treated elsewhere) is observed there, so no ATT(g, t) at that period
    is identified against an untreated outcome.
  • "drop" (default) is byte-identical to the shipped M-125 behavior: unsupported
    periods are removed before the solve and the reduction is warned. Measured on the
    {3,5,8} all-eventually-treated builder: overall_att 1.5416, cells
    (3,3..7),(5,5..7), 1470 of 1890 rows retained; the drop-warning text is
    unchanged against the pre-change source.
  • "error" refuses with ValueError at the top of the filter block, before the
    survey_design= refusal and before any row is removed, naming the periods, the
    would-be-dropped observation count and the cause. It is not gated on
    rank_deficient_action: the message is identical across warn/error/silent
    and no comparison-support, reference-movement, zero-cell or rank warning precedes
    it (the pre-filter outcome-fit hint, which reads only the outcome column, can
    legitimately fire first and is pinned as the documented exception).
  • Why refuse rather than skip the filter: measured with the filter disabled, every
    predicate branch and every rank mode refuses anyway, because the cells at an
    unsupported period sum to that period's time indicator; skipping can only yield a
    rank-deficient or completeness-gate refusal with a vaguer message, never a fit.
  • Cause accuracy: support is weight-aware, so a period can be unsupported
    structurally (no eligible row observed) or by zero survey weight (eligible
    rows observed, all at weight 0). A shared helper _comparison_support_cause
    names the right one for both the warning and the refusal; single-cause sentences
    are byte-identical to the pre-existing warning, and only mixed frames carry
    explicit per-cause period lists. The remedy is conditional on survey_design=
    (the default also refuses under a survey design, and the message carries the
    PSU/stratum caveat of the survey refusal).
  • Two post-solve gate messages that presumed the drop had happened are reworded to
    stay true under both modes. No estimate, weight, variance or aggregation changes
    under either value; no results field is added (a successful "error" fit runs
    code identical to "drop"; the omission is recorded in the ledger row).
  • Ledger row M-147 (introduced_in: 4.0, terminal) plus the snapshot bumps
    (tests/test_v4_matrix.py, docs/v4-design.md), the migration-guide bullet, the
    M-125 NOT-INCLUDED clause amended, REGISTRY notes (new note plus two in-place
    qualifications), llms-full.txt, llms-autonomous.txt (and its new
    doc-deps.yaml entry), the API reference note, the ETWFE tutorial
    (markdown-only), TODO.md (row deleted; survey row's exception-type wording
    scoped), and a changelog fragment.
  • Test infrastructure: tests/test_doc_snippets.py now executes snippets inside
    tmp_path, so the API docs' bare savefig('<name>.png') calls no longer write
    PNGs into the checkout. Four generated PNG artifacts were removed.

Methodology references (required if estimator / math changes)

  • Method name(s): Wooldridge extended two-way fixed effects (ETWFE), per-period
    comparison support on all-eventually-treated panels (the cell half of W2025
    Section 5.4, Eqs. 5.13-5.15).
  • Paper / source link(s): Wooldridge, J. M. (2025). Two-way fixed effects, the
    two-way Mundlak regression, and difference-in-differences estimators. Empirical
    Economics
    , 69(5), 2545-2587. https://doi.org/10.1007/s00181-025-02807-z.
    In-repo review: docs/methodology/papers/wooldridge-2025-review.md (Section
    5.4 notes). External anchor: Stata jwdid all-treated golden block
    (tests/test_etwfe_cs_stata_parity.py::TestAllEventuallyTreatedVsStataJwdid),
    unchanged and passing.
  • Any intentional deviations from the source (and why): None new. The default
    path is unchanged and still implements the Section 5.4 mechanic; "error" is an
    opt-in refusal that estimates nothing. The pre-existing, REGISTRY-documented
    deviation stands: Stata jwdid performs the same reduction silently, this
    library reports it. The absence of a "keep the rows" mode is a deliberate
    design decision recorded in the REGISTRY note and ledger row with the measured
    evidence.

Validation

  • Tests added/updated: tests/test_wooldridge.py (new class
    TestUnsupportedPeriodAction, 10 test functions / 21 collected cases: constructor contract incl. transactional
    set_params; 4-branch x 3-rank-mode refusal matrix; zero-weight, mixed-cause and
    all-weight-one survey panels on OLS/logit/Poisson; no-op invariance; ordering
    ahead of the survey refusal; default byte-identity; refusal replacing the
    completeness gate; warning/refusal wording lockstep; the binary-outcome hint as
    the one warning that may precede the refusal), two inline panels extracted into
    shared builders with existing assertions unchanged, one default-construction
    assertion; tests/test_v4_matrix.py (M-147 snapshot + contract tally);
    tests/test_doc_snippets.py (CWD isolation).
  • Checks executed during development (local .venv, Python 3.14):
    full default suite 14095 passed / 224 skipped / 253 deselected (run before the
    subsequent documentation and test revisions); after those revisions, tests/test_wooldridge.py
    • tests/test_v4_matrix.py + tests/test_tracking_files.py 605 passed /
      24 skipped, and tests/test_doc_snippets.py + tests/test_tracking_files.py
      206 passed / 7 skipped with no PNG produced; tests/test_wooldridge.py under
      both DIFF_DIFF_BACKEND values (282 passed each); contract/ledger/docs/guide
      suites (test_base_estimator, test_naming_guard, test_v4_matrix,
      test_tracking_files, test_changelog_fragments, test_docs_ia,
      test_doc_deps_integrity, test_notebook_md_extract, test_guides) 988 passed /
      67 skipped; tests/test_etwfe_cs_stata_parity.py 33 passed; premerge_scan.py
      0 findings; changelog_compile.py check OK; secret scan clean; ruff and black
      clean.
  • Backtest / simulation / notebook evidence (if applicable):
    docs/tutorials/16_wooldridge_etwfe.ipynb executed via pytest --nbmake
    (1 passed, no outputs written back; the edit is markdown-only and its stored
    drop-warning output remains valid under the unchanged default). In-memory
    measurements behind the design (filter disabled: every branch/rank mode
    refuses; zero-weight and mixed-cause reachability on both predicate branches)
    are recorded in the REGISTRY note and the M-147 ledger notes.
  • Limitations: the project-configured mypy diff_diff (Python 3.10 target)
    stops before checking any project file in this frozen environment, on the
    NumPy 2.5.2 stubs' Python 3.12 type statements (also reproduced on unchanged source; the frozen lint workflow pins NumPy 2.4.5). Local type-check
    evidence comes from an unchanged configuration run with a command-line
    --python-version 3.12 override: no issues in 108 source files. That is
    evidence for the changed code, not a substitute for the pinned CI lint lane.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes (content and filename secret scans run on
    the working tree; nothing flagged).

Changelog

  • changelog.d/ fragment added (or N/A - no user-visible change): Yes,
    changelog.d/20260905-etwfe-unsupported-period-action.md (Added: M-147;
    Internal: doc-snippet CWD isolation).

@igerber igerber added ready-for-ci Triggers CI test workflows and removed ready-for-ci Triggers CI test workflows labels Sep 5, 2026
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

Overall Assessment: ✅ Looks good

Executive Summary

  • Affects Wooldridge ETWFE per-period comparison-support handling only.
  • Behavior aligns with W2025 §5.4/Eq. 5.15 and the Methodology Registry.
  • Default estimates, weighting, variance, SEs, and aggregation remain unchanged.
  • Constructor validation, parameter propagation, survey weights, rank modes, and empty-support refusal are covered.
  • No P0/P1/P2 findings. One documented P3 metadata exception exists.

Methodology

  • Severity: P3 — informational. "error" is a documented library refusal rather than a new estimator variant; it produces no estimate. The default "drop" behavior remains consistent with W2025’s unidentified final-period treatment cells (paper review, implementation, Registry).
    Impact: No mathematical, weighting, variance, or identification change.
    Concrete fix: None required.

Code Quality

  • Severity: None. Cause classification is centralized and the new branch fails before mutating the sample (wooldridge.py).
    Impact: No misleading partial result or inference path identified.
    Concrete fix: None.

Performance

  • Severity: None. The added masks and period classification are linear in sample size and reuse existing arrays.
    Impact: Negligible additional fit overhead.
    Concrete fix: None.

Maintainability

  • Severity: P3 — informational. unsupported_period_action is intentionally absent from WooldridgeDiDResults, contrary to the general new-parameter convention, but this exception is explicitly documented in the Registry (Registry, results metadata).
    Impact: Successful result objects do not record whether "drop" or "error" was configured, although successful estimates are identical.
    Concrete fix: None required under the documented deviation.

Tech Debt

  • Severity: P3 — informational. Survey-domain estimation remains unsupported and is explicitly tracked in TODO.md. Both modes fail closed rather than calculating variance from an altered survey design.
    Impact: Reduced capability, not silent statistical error.
    Concrete fix: Follow the tracked domain-estimation work; no PR-blocking change.

Security

  • Severity: None. No secret material, unsafe execution path, or sensitive-data exposure identified.
    Impact: None.
    Concrete fix: None.

Documentation / Tests

  • Severity: None. A changelog fragment is present, and tests cover validation, transactional set_params, OLS/logit/Poisson branches, all rank modes, structural/zero-weight/mixed causes, survey ordering, no-op invariance, and default behavior (tests).
    Impact: Appropriate coverage for the behavior change.
    Concrete fix: None.

Runtime tests could not be executed in the review container because pytest and NumPy are unavailable; Python syntax and notebook JSON validation passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant