diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-05-29 12:52:29 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-05-29 12:52:29 -0500 |
| commit | 385a3c0fc97c517bc7361bbb86ca6a134ca198c0 (patch) | |
| tree | 53ffa1de9f6a38ee2e08587d935b919f9058fbc1 /notes/01_theory_notes.md | |
| parent | 96e201556ac94057a4a5c8864cf422ad43d72d58 (diff) | |
Add trajectory gap distribution bridge
Diffstat (limited to 'notes/01_theory_notes.md')
| -rw-r--r-- | notes/01_theory_notes.md | 28 |
1 files changed, 28 insertions, 0 deletions
diff --git a/notes/01_theory_notes.md b/notes/01_theory_notes.md index b22e535..d0ac8da 100644 --- a/notes/01_theory_notes.md +++ b/notes/01_theory_notes.md @@ -492,3 +492,31 @@ If \(\epsilon_t(B)\) is approximately Gaussian under random \(B\), then: \] This should be treated as a bridge approximation, not as a primary architecture-only theorem. + +### First Empirical Bridge Variant + +For a concrete first trajectory-distribution predictor, ignore the Hessian propagation factors and integrate the FA/BP gradient mismatch along the BP path: + +\[ +\widehat{\delta\theta}_T(B) += +-\eta +\sum_{t<T} +\left[ +g_t^{\mathrm{FA}}(\theta_t^{\mathrm{BP}};B) +- +g_t^{\mathrm{BP}}(\theta_t^{\mathrm{BP}}) +\right]. +\] + +Then predict the final FA/BP loss gap by evaluating the loss at the BP endpoint plus this bridge deviation: + +\[ +\widehat{\Delta L}_T(B) += +L(\theta_T^{\mathrm{BP}}+\widehat{\delta\theta}_T(B)) +- +L(\theta_T^{\mathrm{BP}}). +\] + +This is expected to be misscaled when curvature and contraction along the BP path are important. It is still useful as a distributional shape test: if the standardized \(\widehat{\Delta L}_T(B)\) distribution matches the standardized empirical \(\Delta L_T(B)\) distribution, then the bridge captures the random-feedback shape but not yet the absolute scale. A stronger next theorem should include a curvature or response operator replacing the identity propagation used here. |
