Add trajectory ensemble validation

1 files changed, 52 insertions, 0 deletions

diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md
index c44e0d0..896961b 100644
--- a/notes/02_experiment_notes.md
+++ b/notes/02_experiment_notes.md
@@ -342,3 +342,55 @@ Per-seed summary:
 This is only a smoke trajectory, not yet an ensemble result. It verifies that the logging pipeline can capture loss gaps, surrogate-gradient alignment, and weight-feedback alignment \(Q_l(t)\) from the same run.
 
 Important metric note: full-model gradient cosine can be inflated by the output layer, whose gradient is identical under BP and FA. Hidden-only gradient cosine is a sharper metric for feedback-induced mismatch.
+
+## Trajectory Ensemble Run Log
+
+Script:
+
+```bash
+python scripts/trajectory_ensemble.py --architectures 16,16 24,24 32,32 48,48 24,24,24 --samples 256 --steps 200 --lr 0.02 --eval-every 20 --feedback-runs 40 --data-seed 20 --init-seed 30 --feedback-seed-start 1000 --plot
+```
+
+Setup:
+
+- architectures: `16,16`, `24,24`, `32,32`, `48,48`, `24,24,24`
+- feedback seeds per architecture: `40`
+- total FA trajectories: `200`
+- synthetic full-batch regression
+- shared BP baseline per architecture
+
+Architecture-level results:
+
+| architecture | BP final loss | FA gap mean | FA gap std | gap min | gap max | final hidden cosine mean | final capacity mean |
+|---|---:|---:|---:|---:|---:|---:|---:|
+| `h16x16` | `0.549502` | `0.029880` | `0.054179` | `-0.111327` | `0.113590` | `0.305739` | `6.435878` |
+| `h24x24` | `0.560063` | `0.187094` | `0.064862` | `0.059168` | `0.316122` | `0.256078` | `7.749068` |
+| `h32x32` | `0.427190` | `0.068749` | `0.023786` | `0.029412` | `0.124503` | `0.190019` | `8.956942` |
+| `h48x48` | `0.349687` | `0.078056` | `0.018102` | `0.032782` | `0.113509` | `0.196928` | `15.784286` |
+| `h24x24x24` | `0.336629` | `0.066436` | `0.020885` | `0.010641` | `0.106398` | `0.008886` | `4.814671` |
+
+Pooled result:
+
+- total FA runs: `200`
+- final gap mean: `0.086042897`
+- final gap std: `0.067060324`
+- final gap min: `-0.11132673`
+- final gap max: `0.31612214`
+- negative-gap FA runs: `9/200`, all in `h16x16`
+
+Correlation takeaways:
+
+- Pooled correlations are weak because architecture-level differences dominate and confound the relationship between capacity proxies and gap.
+- Within architecture, final hidden-gradient alignment is consistently predictive: larger final hidden cosine corresponds to smaller final FA/BP gap.
+- Strongest within-architecture Spearman correlations with final gap:
+  - `h16x16`: final hidden cosine, \(\rho=-0.4587\), \(p=0.00291\)
+  - `h24x24`: final capacity, \(\rho=-0.4432\), \(p=0.00419\)
+  - `h32x32`: final hidden cosine, \(\rho=-0.4477\), \(p=0.00377\)
+  - `h48x48`: final hidden cosine, \(\rho=-0.5180\), \(p=0.000618\)
+  - `h24x24x24`: final hidden cosine, \(\rho=-0.7235\), \(p=1.34e-7\)
+
+Interpretation:
+
+- Static \(Q\) and capacity proxies alone do not explain pooled loss gaps across architectures.
+- Same-architecture seed variation is much better explained by trajectory-level hidden-gradient alignment.
+- This supports the current paper framing: static distributional capacity bounds quantify the initial burden, while trajectory bridge metrics are needed to explain realized FA/BP loss gaps.