summaryrefslogtreecommitdiff
path: root/notes/01_theory_notes.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-05-29 12:52:29 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-05-29 12:52:29 -0500
commit385a3c0fc97c517bc7361bbb86ca6a134ca198c0 (patch)
tree53ffa1de9f6a38ee2e08587d935b919f9058fbc1 /notes/01_theory_notes.md
parent96e201556ac94057a4a5c8864cf422ad43d72d58 (diff)
Add trajectory gap distribution bridge
Diffstat (limited to 'notes/01_theory_notes.md')
-rw-r--r--notes/01_theory_notes.md28
1 files changed, 28 insertions, 0 deletions
diff --git a/notes/01_theory_notes.md b/notes/01_theory_notes.md
index b22e535..d0ac8da 100644
--- a/notes/01_theory_notes.md
+++ b/notes/01_theory_notes.md
@@ -492,3 +492,31 @@ If \(\epsilon_t(B)\) is approximately Gaussian under random \(B\), then:
\]
This should be treated as a bridge approximation, not as a primary architecture-only theorem.
+
+### First Empirical Bridge Variant
+
+For a concrete first trajectory-distribution predictor, ignore the Hessian propagation factors and integrate the FA/BP gradient mismatch along the BP path:
+
+\[
+\widehat{\delta\theta}_T(B)
+=
+-\eta
+\sum_{t<T}
+\left[
+g_t^{\mathrm{FA}}(\theta_t^{\mathrm{BP}};B)
+-
+g_t^{\mathrm{BP}}(\theta_t^{\mathrm{BP}})
+\right].
+\]
+
+Then predict the final FA/BP loss gap by evaluating the loss at the BP endpoint plus this bridge deviation:
+
+\[
+\widehat{\Delta L}_T(B)
+=
+L(\theta_T^{\mathrm{BP}}+\widehat{\delta\theta}_T(B))
+-
+L(\theta_T^{\mathrm{BP}}).
+\]
+
+This is expected to be misscaled when curvature and contraction along the BP path are important. It is still useful as a distributional shape test: if the standardized \(\widehat{\Delta L}_T(B)\) distribution matches the standardized empirical \(\Delta L_T(B)\) distribution, then the bridge captures the random-feedback shape but not yet the absolute scale. A stronger next theorem should include a curvature or response operator replacing the identity propagation used here.