diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-05-28 23:11:40 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-05-28 23:11:40 -0500 |
| commit | 0a3cc65f54d4ffc90b0e73f1f8713820352b27bf (patch) | |
| tree | 927f342b96e7ea71c444776d722c8c46e57de4ce /notes | |
| parent | 89ebbd7ba50321e790520522939a941ff5ea96b7 (diff) | |
Add hidden gradient alignment metric
Diffstat (limited to 'notes')
| -rw-r--r-- | notes/02_experiment_notes.md | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/notes/02_experiment_notes.md b/notes/02_experiment_notes.md index 1d641e1..c44e0d0 100644 --- a/notes/02_experiment_notes.md +++ b/notes/02_experiment_notes.md @@ -331,6 +331,7 @@ Result: - BP final loss: `0.60596695` - FA final gap to BP: mean `0.2032423`, min `0.16939124`, max `0.25930484` - FA final BP/FA gradient cosine: mean `0.31281339`, min `0.29023733`, max `0.35150802` +- FA final hidden-only BP/FA gradient cosine: mean `-0.0070103243`, min `-0.13335294`, max `0.072182807` Per-seed summary: @@ -339,3 +340,5 @@ Per-seed summary: - seed `52`: final loss `0.78699777`, final gap `0.18103082`, initial \(Q\) mean `0.02377742`, final \(Q\) mean `0.00360508` This is only a smoke trajectory, not yet an ensemble result. It verifies that the logging pipeline can capture loss gaps, surrogate-gradient alignment, and weight-feedback alignment \(Q_l(t)\) from the same run. + +Important metric note: full-model gradient cosine can be inflated by the output layer, whose gradient is identical under BP and FA. Hidden-only gradient cosine is a sharper metric for feedback-induced mismatch. |