diff options
Diffstat (limited to 'notes/02_experiment_notes.md')
| -rw-r--r-- | notes/02_experiment_notes.md | 50 |
1 files changed, 50 insertions, 0 deletions
diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md index c7e7168..03666c6 100644 --- a/notes/02_experiment_notes.md +++ b/notes/02_experiment_notes.md @@ -552,3 +552,53 @@ Interpretation: - Static \(Q\) and capacity proxies alone do not explain pooled loss gaps across architectures. - Same-architecture seed variation is much better explained by trajectory-level hidden-gradient alignment. - This supports the current paper framing: static distributional capacity bounds quantify the initial burden, while trajectory bridge metrics are needed to explain realized FA/BP loss gaps. + +## Trajectory Gap Distribution Bridge Run Log + +Script for `h16x16`: + +```bash +python scripts/trajectory_gap_distribution.py --hidden-widths 16 16 --samples 128 --steps 60 --lr 0.02 --feedback-runs 1000 --data-seed 7 --init-seed 8 --feedback-seed-start 10000 --outdir outputs/trajectory_gap_distribution/h16x16_1000 --plot +``` + +Script for `h24x24`: + +```bash +python scripts/trajectory_gap_distribution.py --hidden-widths 24 24 --samples 128 --steps 60 --lr 0.02 --feedback-runs 500 --data-seed 7 --init-seed 8 --feedback-seed-start 20000 --outdir outputs/trajectory_gap_distribution/h24x24_500 --plot +``` + +Bridge predictor: + +\[ +\widehat{\delta\theta}_T(B) += +-\eta +\sum_{t<T} +\left[ +g_{\mathrm{FA}}(\theta_t^{\mathrm{BP}};B) +- +g_{\mathrm{BP}}(\theta_t^{\mathrm{BP}}) +\right], +\] + +\[ +\widehat{\Delta L}_T(B) += +L(\theta_T^{\mathrm{BP}}+\widehat{\delta\theta}_T(B)) +- +L(\theta_T^{\mathrm{BP}}). +\] + +Results: + +| architecture | runs | BP final loss | empirical gap mean | bridge gap mean | empirical std | bridge std | raw KS | standardized KS | standardized p | moment-matched W1 | Spearman | +|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:| +| `h16x16` | `1000` | `0.604310` | `0.120109` | `0.318266` | `0.041817` | `0.081883` | `0.917` | `0.056` | `0.0869` | `0.00362` | `0.49288` | +| `h24x24` | `500` | `0.572197` | `0.145215` | `0.432834` | `0.055733` | `0.108192` | `0.944` | `0.056` | `0.4135` | `0.00356` | `0.39455` | + +Interpretation: + +- The raw bridge predictor overestimates absolute gap scale by roughly \(2\)-\(3\times\), so the current identity-propagation bridge is not yet a final calibrated theory for \(\Delta L_T\). +- The standardized distributions match surprisingly well: both architectures have standardized KS `0.056`, with non-rejected two-sample tests at these sample sizes. +- The paired seed-level rank signal is nontrivial: Spearman is `0.49288` for `h16x16` and `0.39455` for `h24x24`. +- Current conclusion: the BP-path mismatch bridge captures much of the random-feedback distributional shape but lacks the curvature/contraction operator needed to predict absolute scale. |