From 225459f7ddfcd3c8d89f0515a0d766bf2a15c848 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Wed, 22 Jul 2026 13:21:20 -0500 Subject: algorithm: calibrate hierarchical feedback parameter subspaces --- THEORY.md | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) (limited to 'THEORY.md') diff --git a/THEORY.md b/THEORY.md index 93ef88c..1c4f5d0 100644 --- a/THEORY.md +++ b/THEORY.md @@ -188,6 +188,67 @@ mean has cosine `0.96928` and norm ratio `1.04888` to the exact A/G target. Those mechanics do not establish task utility; a separately frozen gate is still required. +### Perturbing the hierarchical feedback-parameter subspace + +The hierarchical oracle shows that an early field needs the spatial child +teaching map rather than only the output-error vector. For a residual-DAG edge +`e`, let `s_e` be the locally gated and BatchNorm-transformed child field and +let the feedback contribution at its parent be + +```text +p_e = conv_transpose(s_e, Q_e). +``` + +The complete predicted parent field `p` also includes the exact +parameter-free shortcut contribution. Draw an independent Rademacher tensor +`Xi_e` with the shape of every feedback convolution and intervene at each +edge's parent along + +```text +d_e = conv_transpose(s_e, Xi_e). +``` + +The readout-feedback matrix is treated identically, with +`d_out=(c Xi_out^T)/S` at the final spatial map. All edge fields are injected +in one antithetic pair. If `D` is the derivative of the summed minibatch loss, +independence across parameter tensors gives + +```text +E[-D Xi_e / (B S_parent)] + = corr(q_parent, s_e) / (B S_parent), +q_parent = -d ell / d h_parent. +``` + +All other edges contribute zero-mean cross terms. The locally available +prediction moment is + +```text +corr(p_parent, s_e) / (B S_parent), +``` + +where `p_parent` includes both convolutional and shortcut feedback. Therefore + +```text +Delta Q_e = eta_A [target_moment_e - prediction_moment_e] +``` + +is an unbiased stochastic delta rule for the normalized squared parent-field +error. It uses two task-loss queries per direction, no copied forward tensor, +and no reverse differentiation. Unlike V2/V3, its perturbation family already +contains the spatial child teaching context. Unlike weight mirroring, its +target is task-causal credit rather than reconstruction of the forward +convolution. Recursive feedback calibration and feedback convolutions are not +claimed as novel by themselves; this is an engineering prerequisite for a +hierarchical innovation model and requires direct HFA, learned-FA/mirroring, +and BurstCCN comparisons. + +The executable audit checks the complete BatchNorm-coupled antithetic JVP to +`6.2e-10` relative error. Under an audit-only symmetric feedback copy, the +predicted Q/R moments equal the exact causal delta-rule targets to `2.3e-16` +relative error. Actual training never performs that copy. These identities +establish mechanics only; a frozen-forward causal-capture gate must pass before +any task-scale endpoint is opened. + ## 2. What is sufficient for a descent step Flatten all forward parameters into `theta`, let `p = grad L(theta)`, and let -- cgit v1.2.3