summaryrefslogtreecommitdiff
path: root/THEORY.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 13:21:20 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 13:21:20 -0500
commit225459f7ddfcd3c8d89f0515a0d766bf2a15c848 (patch)
tree329ec70d95000f00b358db70f8d6273cb6d9d8a9 /THEORY.md
parentf37644eb452f0eb3d7f368740f8b39493f11b2aa (diff)
algorithm: calibrate hierarchical feedback parameter subspaces
Diffstat (limited to 'THEORY.md')
-rw-r--r--THEORY.md61
1 files changed, 61 insertions, 0 deletions
diff --git a/THEORY.md b/THEORY.md
index 93ef88c..1c4f5d0 100644
--- a/THEORY.md
+++ b/THEORY.md
@@ -188,6 +188,67 @@ mean has cosine `0.96928` and norm ratio `1.04888` to the exact A/G target.
Those mechanics do not establish task utility; a separately frozen gate is
still required.
+### Perturbing the hierarchical feedback-parameter subspace
+
+The hierarchical oracle shows that an early field needs the spatial child
+teaching map rather than only the output-error vector. For a residual-DAG edge
+`e`, let `s_e` be the locally gated and BatchNorm-transformed child field and
+let the feedback contribution at its parent be
+
+```text
+p_e = conv_transpose(s_e, Q_e).
+```
+
+The complete predicted parent field `p` also includes the exact
+parameter-free shortcut contribution. Draw an independent Rademacher tensor
+`Xi_e` with the shape of every feedback convolution and intervene at each
+edge's parent along
+
+```text
+d_e = conv_transpose(s_e, Xi_e).
+```
+
+The readout-feedback matrix is treated identically, with
+`d_out=(c Xi_out^T)/S` at the final spatial map. All edge fields are injected
+in one antithetic pair. If `D` is the derivative of the summed minibatch loss,
+independence across parameter tensors gives
+
+```text
+E[-D Xi_e / (B S_parent)]
+ = corr(q_parent, s_e) / (B S_parent),
+q_parent = -d ell / d h_parent.
+```
+
+All other edges contribute zero-mean cross terms. The locally available
+prediction moment is
+
+```text
+corr(p_parent, s_e) / (B S_parent),
+```
+
+where `p_parent` includes both convolutional and shortcut feedback. Therefore
+
+```text
+Delta Q_e = eta_A [target_moment_e - prediction_moment_e]
+```
+
+is an unbiased stochastic delta rule for the normalized squared parent-field
+error. It uses two task-loss queries per direction, no copied forward tensor,
+and no reverse differentiation. Unlike V2/V3, its perturbation family already
+contains the spatial child teaching context. Unlike weight mirroring, its
+target is task-causal credit rather than reconstruction of the forward
+convolution. Recursive feedback calibration and feedback convolutions are not
+claimed as novel by themselves; this is an engineering prerequisite for a
+hierarchical innovation model and requires direct HFA, learned-FA/mirroring,
+and BurstCCN comparisons.
+
+The executable audit checks the complete BatchNorm-coupled antithetic JVP to
+`6.2e-10` relative error. Under an audit-only symmetric feedback copy, the
+predicted Q/R moments equal the exact causal delta-rule targets to `2.3e-16`
+relative error. Actual training never performs that copy. These identities
+establish mechanics only; a frozen-forward causal-capture gate must pass before
+any task-scale endpoint is opened.
+
## 2. What is sufficient for a descent step
Flatten all forward parameters into `theta`, let `p = grad L(theta)`, and let