diff options
Diffstat (limited to 'notes/37_wrapup_corrections.md')
| -rw-r--r-- | notes/37_wrapup_corrections.md | 79 |
1 files changed, 79 insertions, 0 deletions
diff --git a/notes/37_wrapup_corrections.md b/notes/37_wrapup_corrections.md new file mode 100644 index 0000000..671c237 --- /dev/null +++ b/notes/37_wrapup_corrections.md @@ -0,0 +1,79 @@ +# Wrap-Up Corrections (pre-writing audit fixes) + +Two integrity issues found in the pre-writing audit (2026-06-09) and fixed. +Both were flagged as cleanup items in note 35; this note records the fixes and +the corrected numbers. The evidence ledger (note 36) already uses them. + +## 1. e0-distribution sampler was partly circular (fixed) + +**Problem.** `scripts/actual_fa_initial_erosion_distribution.py` sampled the +erosion via the analytic linear form `speed_FA = output_speed + <C, B>`. Since +a linear functional of Gaussian `B` is Gaussian by construction, the KS test +against the Gaussian theory was close to tautological — it validated the +implementation of `C`, not the claim about real FA. + +**Fix.** `sample_erosions` now runs the **real FA backward pass** for every +feedback draw (`delta_hidden = (delta_out @ B^T) * gate`, then +`<g_hidden^BP, g_hidden^FA>`). The analytic form is demoted to an internal +consistency check: the per-draw deviation between the real backward and +`<C, B>` is tracked and reported as `linear_identity_max_err`. + +**Result (rerun, same seeds).** Identical numbers to the published table in +note 32 — as they must be, since the linear identity holds exactly: +`linear_identity_max_err <= 8.9e-16` (float64 roundoff) across all panels. +KS p-values 0.13-0.999 over widths {16,32,64,128} x 3 inits x 4096 draws. +The figure is now non-circular: histogram from real backward, density from the +theorem. Old outputs preserved in +`outputs/actual_fa_initial_erosion_distribution_pre_fix_backup/`. + +For one hidden layer the linearity in `B` is exact algebra, so the empirical +content of this experiment is (i) the identity holds for the real backward +implementation and (ii) mean/std match the explicit formulas. Both now tested. + +## 2. Closed-form comparison was computed on mismatched data and init (fixed) + +**Problem.** `scripts/closed_form_soft_ramp.py` hard-coded the dense-sweep +gaps (copied from note 22) and evaluated the closed form with its own defaults +`data_seed=123`, `init_seed=0`. The dense T=30000 sweep actually used +`data_seed=0` and init seed `10000` (= `10_000 + init_seed_offset`, +`downstream_capacity_sweep.main`). So prediction and measurement lived on +different data and different initializations, and the reported +`corr(log pred, log measured) = 0.977` (notes 34/35) was partly luck. + +**Fix.** The script now loads the dense run's `summary.json` (config) and +`width_summary.csv` (measured gaps + stds) from `--dense-dir`, reproduces the +exact training data, evaluates the closed form at the **matched init seed**, +and additionally at 4 extra init seeds on the same data to expose the +init-to-init variability of the no-fit prediction. + +**Corrected numbers (rerun).** + +| quantity | value | +|---|---| +| corr(log pred, log measured), matched single init | **0.933** | +| corr(log pred, log measured), 5-init geometric mean | **0.982** | +| corr(lam_min, -log measured), matched init | 0.948 | +| corr(lam_min mean over inits, -log measured) | **0.987** | +| measured log-linearity (unchanged, data-only) | R^2 = 0.993 | +| ensemble-mean lam_min(w) | 0.026 -> 0.763 (monotone) | +| rho across widths (5-init mean) | 0.68-0.79 | + +Reading: a single init's frozen `lam_min(w)` is noisy across widths, while the +T=30000 measured ramp is smooth (drift + feedback-averaging smooth it), so the +matched-single-init correlation is lower (0.933). The init-ensemble closed +form — still zero fitted parameters — tracks the ramp at 0.982 and isolates +`lam_min` as the driver at 0.987. + +**For the paper:** report 0.982 (ensemble) and 0.933 (matched) and retire +0.977; the figure now shows measured (error bars), matched-init curve, +ensemble geometric mean, and the init-seed band. + +## 3. Related cleanup executed with this audit + +- `scripts/alignment_recovery_dynamics.py`: superseded probe (note 35); + its divide-by-zero guard was fixed earlier but the rerun was never completed + and its outputs are empty. Removed; its one useful datum (independent t=0 + anchor `E_B[e_0] = 0.4026 +/- 0.0066` vs `hidden_share = 0.4071`) is + recorded here and in the ledger. +- Notes 00/03 carry a superseded banner pointing at notes 35/36. +- Headline-number corrections appended to notes 34/35. |