Add trajectory gap distribution bridge

3 files changed, 84 insertions, 0 deletions

diff --git a/notes/01_theory_notes.md b/notes/01_theory_notes.md
index b22e535..d0ac8da 100644
--- a/notes/01_theory_notes.md
+++ b/notes/01_theory_notes.md
@@ -492,3 +492,31 @@ If \(\epsilon_t(B)\) is approximately Gaussian under random \(B\), then:
 \]
 
 This should be treated as a bridge approximation, not as a primary architecture-only theorem.
+
+### First Empirical Bridge Variant
+
+For a concrete first trajectory-distribution predictor, ignore the Hessian propagation factors and integrate the FA/BP gradient mismatch along the BP path:
+
+\[
+\widehat{\delta\theta}_T(B)
+=
+-\eta
+\sum_{t<T}
+\left[
+g_t^{\mathrm{FA}}(\theta_t^{\mathrm{BP}};B)
+-
+g_t^{\mathrm{BP}}(\theta_t^{\mathrm{BP}})
+\right].
+\]
+
+Then predict the final FA/BP loss gap by evaluating the loss at the BP endpoint plus this bridge deviation:
+
+\[
+\widehat{\Delta L}_T(B)
+=
+L(\theta_T^{\mathrm{BP}}+\widehat{\delta\theta}_T(B))
+-
+L(\theta_T^{\mathrm{BP}}).
+\]
+
+This is expected to be misscaled when curvature and contraction along the BP path are important. It is still useful as a distributional shape test: if the standardized \(\widehat{\Delta L}_T(B)\) distribution matches the standardized empirical \(\Delta L_T(B)\) distribution, then the bridge captures the random-feedback shape but not yet the absolute scale. A stronger next theorem should include a curvature or response operator replacing the identity propagation used here.
diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md
index c7e7168..03666c6 100644
--- a/notes/02_experiment_notes.md
+++ b/notes/02_experiment_notes.md
@@ -552,3 +552,53 @@ Interpretation:
 - Static \(Q\) and capacity proxies alone do not explain pooled loss gaps across architectures.
 - Same-architecture seed variation is much better explained by trajectory-level hidden-gradient alignment.
 - This supports the current paper framing: static distributional capacity bounds quantify the initial burden, while trajectory bridge metrics are needed to explain realized FA/BP loss gaps.
+
+## Trajectory Gap Distribution Bridge Run Log
+
+Script for `h16x16`:
+
+```bash
+python scripts/trajectory_gap_distribution.py --hidden-widths 16 16 --samples 128 --steps 60 --lr 0.02 --feedback-runs 1000 --data-seed 7 --init-seed 8 --feedback-seed-start 10000 --outdir outputs/trajectory_gap_distribution/h16x16_1000 --plot
+```
+
+Script for `h24x24`:
+
+```bash
+python scripts/trajectory_gap_distribution.py --hidden-widths 24 24 --samples 128 --steps 60 --lr 0.02 --feedback-runs 500 --data-seed 7 --init-seed 8 --feedback-seed-start 20000 --outdir outputs/trajectory_gap_distribution/h24x24_500 --plot
+```
+
+Bridge predictor:
+
+\[
+\widehat{\delta\theta}_T(B)
+=
+-\eta
+\sum_{t<T}
+\left[
+g_{\mathrm{FA}}(\theta_t^{\mathrm{BP}};B)
+-
+g_{\mathrm{BP}}(\theta_t^{\mathrm{BP}})
+\right],
+\]
+
+\[
+\widehat{\Delta L}_T(B)
+=
+L(\theta_T^{\mathrm{BP}}+\widehat{\delta\theta}_T(B))
+-
+L(\theta_T^{\mathrm{BP}}).
+\]
+
+Results:
+
+| architecture | runs | BP final loss | empirical gap mean | bridge gap mean | empirical std | bridge std | raw KS | standardized KS | standardized p | moment-matched W1 | Spearman |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| `h16x16` | `1000` | `0.604310` | `0.120109` | `0.318266` | `0.041817` | `0.081883` | `0.917` | `0.056` | `0.0869` | `0.00362` | `0.49288` |
+| `h24x24` | `500` | `0.572197` | `0.145215` | `0.432834` | `0.055733` | `0.108192` | `0.944` | `0.056` | `0.4135` | `0.00356` | `0.39455` |
+
+Interpretation:
+
+- The raw bridge predictor overestimates absolute gap scale by roughly \(2\)-\(3\times\), so the current identity-propagation bridge is not yet a final calibrated theory for \(\Delta L_T\).
+- The standardized distributions match surprisingly well: both architectures have standardized KS `0.056`, with non-rejected two-sample tests at these sample sizes.
+- The paired seed-level rank signal is nontrivial: Spearman is `0.49288` for `h16x16` and `0.39455` for `h24x24`.
+- Current conclusion: the BP-path mismatch bridge captures much of the random-feedback distributional shape but lacks the curvature/contraction operator needed to predict absolute scale.
diff --git a/notes/03_paper_outline.md b/notes/03_paper_outline.md
index 6055722..40ab86f 100644
--- a/notes/03_paper_outline.md
+++ b/notes/03_paper_outline.md
@@ -166,6 +166,12 @@ Introduce local approximation:
 
 Use this to motivate trajectory experiments.
 
+First bridge experiment:
+
+- Predict a full loss-gap distribution by integrating FA/BP gradient mismatch along the BP path.
+- Compare raw, standardized, and moment-matched predicted gap distributions against large FA trajectory ensembles.
+- Current empirical status: standardized shape matches better than raw scale, motivating a response-operator refinement.
+
 ## 8. Experiments
 
 Static distribution: