diff options
Diffstat (limited to 'notes')
| -rw-r--r-- | notes/02_experiment_notes.md | 30 | ||||
| -rw-r--r-- | notes/03_paper_outline.md | 6 |
2 files changed, 36 insertions, 0 deletions
diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md index 889d1bd..1d641e1 100644 --- a/notes/02_experiment_notes.md +++ b/notes/02_experiment_notes.md @@ -309,3 +309,33 @@ Result: - \(k=96\): hard loss `24`, theory `24`; soft overlap `24`, theory `24` This validates the redundancy-exhaustion interpretation: hard functional rank remains intact until alignment constraints exceed the redundant dimension \(P-d\), while soft overlap grows linearly as \(kd/P\). + +## Synthetic Trajectory Run Log + +Script: + +```bash +python scripts/trajectory_mlp_fa.py --samples 128 --hidden-widths 24 24 --steps 80 --lr 0.02 --eval-every 10 --feedback-runs 3 --data-seed 3 --init-seed 4 --feedback-seed-start 50 --plot +``` + +Setup: + +- student widths: `[16, 24, 24, 4]` +- synthetic teacher with matching widths +- full-batch MSE +- one BP baseline from the shared initialization +- three FA runs with feedback seeds `50, 51, 52` + +Result: + +- BP final loss: `0.60596695` +- FA final gap to BP: mean `0.2032423`, min `0.16939124`, max `0.25930484` +- FA final BP/FA gradient cosine: mean `0.31281339`, min `0.29023733`, max `0.35150802` + +Per-seed summary: + +- seed `50`: final loss `0.86527179`, final gap `0.25930484`, initial \(Q\) mean `0.00093627`, final \(Q\) mean `0.00281211` +- seed `51`: final loss `0.77535819`, final gap `0.16939124`, initial \(Q\) mean `0.01641749`, final \(Q\) mean `0.04462749` +- seed `52`: final loss `0.78699777`, final gap `0.18103082`, initial \(Q\) mean `0.02377742`, final \(Q\) mean `0.00360508` + +This is only a smoke trajectory, not yet an ensemble result. It verifies that the logging pipeline can capture loss gaps, surrogate-gradient alignment, and weight-feedback alignment \(Q_l(t)\) from the same run. diff --git a/notes/03_paper_outline.md b/notes/03_paper_outline.md index 829226d..a8f07c8 100644 --- a/notes/03_paper_outline.md +++ b/notes/03_paper_outline.md @@ -171,6 +171,12 @@ Trajectory: - \(Q_l(t)\), \(\Gamma_t\), and \(\Delta L_T\). - redundancy-exhaustion transition. +Initial implementation note: + +- Start with synthetic full-batch regression and no-bias MLPs. +- Record same-point gradient alignment by computing BP and FA gradients at the FA weights. +- Treat early trajectory runs as instrumentation checks before drawing distributional conclusions. + ## 9. Discussion Key points: |