diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-23 19:46:56 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-03-23 19:46:56 -0500 |
| commit | 3012cba6032ee04cc0b82c178fbf8df8e47c7d2f (patch) | |
| tree | f78b2cfb1202a7dd08f9ddab08cb122e5264099d /report | |
| parent | 32123cb36ae9521f60c9b6f67458b931b6540ef2 (diff) | |
Add sweep results confirming terminal gradient matching is essential
12-config sweep: no hyperparameter combination recovers useful credit
gradients without terminal gradient matching (best cos ~0.3 early, decays to ~0).
Diffstat (limited to 'report')
| -rw-r--r-- | report/REPORT.md | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/report/REPORT.md b/report/REPORT.md index 2fa8e31..1df77e1 100644 --- a/report/REPORT.md +++ b/report/REPORT.md @@ -30,7 +30,7 @@ Key constraint: **No hidden BP anchor** — intermediate layers never receive ex **Key findings:** - Credit bridge matches state bridge (~0.94 cosine) on the linear system. - Both far exceed DFA, which provides essentially zero directional credit. -- Credit bridge requires **terminal gradient matching** to succeed. Without it, the value function learns correct values but has uninformative gradients (cosine collapses to ~0.03). Terminal gradient matching uses output-layer-local info only (not hidden BP). +- Credit bridge requires **terminal gradient matching** to succeed. Without it, the value function learns correct values but has uninformative gradients (cosine collapses to ~0.03). This was verified across a 12-config hyperparameter sweep — no combination of noise (σ=0.03–1.0), temperature (λ=0.1–1.0), architecture, or learning rate recovers useful gradients without terminal gradient matching. Terminal gradient matching uses output-layer-local info only (not hidden BP). - FM auxiliary provides marginal additional improvement (0.946 vs 0.940).  |
