summaryrefslogtreecommitdiff
path: root/notes/37_wrapup_corrections.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-06-09 15:26:47 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-06-09 15:26:47 -0500
commitff15a76f241e3beb84b203aa6d530e07a0127fd3 (patch)
treec510c145803b7af4fd4a87d7339d973890525751 /notes/37_wrapup_corrections.md
parent719823128945f22e99dbf93eaf657e8c79e42f74 (diff)
Record evidence ledger and wrap-up corrections
Note 36 consolidates the full evidence chain (numbers, figure sources, negative results, claim discipline) as the paper-writing base; note 37 records the two audit fixes; superseded banners on notes 00/03; README index updated. Superseded alignment_recovery_dynamics probe removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Diffstat (limited to 'notes/37_wrapup_corrections.md')
-rw-r--r--notes/37_wrapup_corrections.md79
1 files changed, 79 insertions, 0 deletions
diff --git a/notes/37_wrapup_corrections.md b/notes/37_wrapup_corrections.md
new file mode 100644
index 0000000..671c237
--- /dev/null
+++ b/notes/37_wrapup_corrections.md
@@ -0,0 +1,79 @@
+# Wrap-Up Corrections (pre-writing audit fixes)
+
+Two integrity issues found in the pre-writing audit (2026-06-09) and fixed.
+Both were flagged as cleanup items in note 35; this note records the fixes and
+the corrected numbers. The evidence ledger (note 36) already uses them.
+
+## 1. e0-distribution sampler was partly circular (fixed)
+
+**Problem.** `scripts/actual_fa_initial_erosion_distribution.py` sampled the
+erosion via the analytic linear form `speed_FA = output_speed + <C, B>`. Since
+a linear functional of Gaussian `B` is Gaussian by construction, the KS test
+against the Gaussian theory was close to tautological — it validated the
+implementation of `C`, not the claim about real FA.
+
+**Fix.** `sample_erosions` now runs the **real FA backward pass** for every
+feedback draw (`delta_hidden = (delta_out @ B^T) * gate`, then
+`<g_hidden^BP, g_hidden^FA>`). The analytic form is demoted to an internal
+consistency check: the per-draw deviation between the real backward and
+`<C, B>` is tracked and reported as `linear_identity_max_err`.
+
+**Result (rerun, same seeds).** Identical numbers to the published table in
+note 32 — as they must be, since the linear identity holds exactly:
+`linear_identity_max_err <= 8.9e-16` (float64 roundoff) across all panels.
+KS p-values 0.13-0.999 over widths {16,32,64,128} x 3 inits x 4096 draws.
+The figure is now non-circular: histogram from real backward, density from the
+theorem. Old outputs preserved in
+`outputs/actual_fa_initial_erosion_distribution_pre_fix_backup/`.
+
+For one hidden layer the linearity in `B` is exact algebra, so the empirical
+content of this experiment is (i) the identity holds for the real backward
+implementation and (ii) mean/std match the explicit formulas. Both now tested.
+
+## 2. Closed-form comparison was computed on mismatched data and init (fixed)
+
+**Problem.** `scripts/closed_form_soft_ramp.py` hard-coded the dense-sweep
+gaps (copied from note 22) and evaluated the closed form with its own defaults
+`data_seed=123`, `init_seed=0`. The dense T=30000 sweep actually used
+`data_seed=0` and init seed `10000` (= `10_000 + init_seed_offset`,
+`downstream_capacity_sweep.main`). So prediction and measurement lived on
+different data and different initializations, and the reported
+`corr(log pred, log measured) = 0.977` (notes 34/35) was partly luck.
+
+**Fix.** The script now loads the dense run's `summary.json` (config) and
+`width_summary.csv` (measured gaps + stds) from `--dense-dir`, reproduces the
+exact training data, evaluates the closed form at the **matched init seed**,
+and additionally at 4 extra init seeds on the same data to expose the
+init-to-init variability of the no-fit prediction.
+
+**Corrected numbers (rerun).**
+
+| quantity | value |
+|---|---|
+| corr(log pred, log measured), matched single init | **0.933** |
+| corr(log pred, log measured), 5-init geometric mean | **0.982** |
+| corr(lam_min, -log measured), matched init | 0.948 |
+| corr(lam_min mean over inits, -log measured) | **0.987** |
+| measured log-linearity (unchanged, data-only) | R^2 = 0.993 |
+| ensemble-mean lam_min(w) | 0.026 -> 0.763 (monotone) |
+| rho across widths (5-init mean) | 0.68-0.79 |
+
+Reading: a single init's frozen `lam_min(w)` is noisy across widths, while the
+T=30000 measured ramp is smooth (drift + feedback-averaging smooth it), so the
+matched-single-init correlation is lower (0.933). The init-ensemble closed
+form — still zero fitted parameters — tracks the ramp at 0.982 and isolates
+`lam_min` as the driver at 0.987.
+
+**For the paper:** report 0.982 (ensemble) and 0.933 (matched) and retire
+0.977; the figure now shows measured (error bars), matched-init curve,
+ensemble geometric mean, and the init-seed band.
+
+## 3. Related cleanup executed with this audit
+
+- `scripts/alignment_recovery_dynamics.py`: superseded probe (note 35);
+ its divide-by-zero guard was fixed earlier but the rerun was never completed
+ and its outputs are empty. Removed; its one useful datum (independent t=0
+ anchor `E_B[e_0] = 0.4026 +/- 0.0066` vs `hidden_share = 0.4071`) is
+ recorded here and in the ledger.
+- Notes 00/03 carry a superseded banner pointing at notes 35/36.
+- Headline-number corrections appended to notes 34/35.