diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:21:20 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:21:20 -0500 |
| commit | 225459f7ddfcd3c8d89f0515a0d766bf2a15c848 (patch) | |
| tree | 329ec70d95000f00b358db70f8d6273cb6d9d8a9 /THEORY.md | |
| parent | f37644eb452f0eb3d7f368740f8b39493f11b2aa (diff) | |
algorithm: calibrate hierarchical feedback parameter subspaces
Diffstat (limited to 'THEORY.md')
| -rw-r--r-- | THEORY.md | 61 |
1 files changed, 61 insertions, 0 deletions
@@ -188,6 +188,67 @@ mean has cosine `0.96928` and norm ratio `1.04888` to the exact A/G target. Those mechanics do not establish task utility; a separately frozen gate is still required. +### Perturbing the hierarchical feedback-parameter subspace + +The hierarchical oracle shows that an early field needs the spatial child +teaching map rather than only the output-error vector. For a residual-DAG edge +`e`, let `s_e` be the locally gated and BatchNorm-transformed child field and +let the feedback contribution at its parent be + +```text +p_e = conv_transpose(s_e, Q_e). +``` + +The complete predicted parent field `p` also includes the exact +parameter-free shortcut contribution. Draw an independent Rademacher tensor +`Xi_e` with the shape of every feedback convolution and intervene at each +edge's parent along + +```text +d_e = conv_transpose(s_e, Xi_e). +``` + +The readout-feedback matrix is treated identically, with +`d_out=(c Xi_out^T)/S` at the final spatial map. All edge fields are injected +in one antithetic pair. If `D` is the derivative of the summed minibatch loss, +independence across parameter tensors gives + +```text +E[-D Xi_e / (B S_parent)] + = corr(q_parent, s_e) / (B S_parent), +q_parent = -d ell / d h_parent. +``` + +All other edges contribute zero-mean cross terms. The locally available +prediction moment is + +```text +corr(p_parent, s_e) / (B S_parent), +``` + +where `p_parent` includes both convolutional and shortcut feedback. Therefore + +```text +Delta Q_e = eta_A [target_moment_e - prediction_moment_e] +``` + +is an unbiased stochastic delta rule for the normalized squared parent-field +error. It uses two task-loss queries per direction, no copied forward tensor, +and no reverse differentiation. Unlike V2/V3, its perturbation family already +contains the spatial child teaching context. Unlike weight mirroring, its +target is task-causal credit rather than reconstruction of the forward +convolution. Recursive feedback calibration and feedback convolutions are not +claimed as novel by themselves; this is an engineering prerequisite for a +hierarchical innovation model and requires direct HFA, learned-FA/mirroring, +and BurstCCN comparisons. + +The executable audit checks the complete BatchNorm-coupled antithetic JVP to +`6.2e-10` relative error. Under an audit-only symmetric feedback copy, the +predicted Q/R moments equal the exact causal delta-rule targets to `2.3e-16` +relative error. Actual training never performs that copy. These identities +establish mechanics only; a frozen-forward causal-capture gate must pass before +any task-scale endpoint is opened. + ## 2. What is sufficient for a descent step Flatten all forward parameters into `theta`, let `p = grad L(theta)`, and let |
