algorithm: estimate causal credit in vectorizer space

1 files changed, 56 insertions, 0 deletions

diff --git a/THEORY.md b/THEORY.md
index 8d20659..676d28e 100644
--- a/THEORY.md
+++ b/THEORY.md
@@ -132,6 +132,62 @@ unrepresentable spatial components.  It remains a variance reduction, not a
 guarantee of useful feedback or stable end-to-end optimization; those require
 a separately frozen empirical gate.
 
+### Perturbing the vectorizer parameter subspace
+
+The channel-moment estimator still samples one independent base/gate
+coefficient for every example and channel and only afterward regresses those
+noisy coefficients on the output context.  The quantity the local delta rule
+actually needs is much smaller.  For
+
+```text
+p_ics = (A c_i)_c + t_ics (G c_i)_c
+```
+
+define the normalized prediction objective
+
+```text
+J(A,G) = (1 / 2B) sum_i mean_s ||p_i - q_i||^2,
+q_i = -d ell_i / dh_i.
+```
+
+Its causal target terms are matrices, not per-example coefficient fields:
+
+```text
+U_A = (1/B) sum_i mean_s(q_ics) c_i^T,
+U_G = (1/B) sum_i mean_s(q_ics t_ics) c_i^T.
+```
+
+Draw independent Rademacher matrices `Xi_A,Xi_G` with the same shapes as
+`A,G`, independently across layers, and intervene along
+
+```text
+d_ics = ((Xi_A c_i)_c + t_ics (Xi_G c_i)_c) / sqrt(2).
+```
+
+Let `D` be the centered derivative of the summed minibatch loss under all
+layer interventions. Since `D=-sum_ics q_ics d_ics`, independence gives
+
+```text
+Uhat_A = -sqrt(2) D Xi_A / (B S),     E[Uhat_A] = U_A,
+Uhat_G = -sqrt(2) D Xi_G / (B S),     E[Uhat_G] = U_G.
+```
+
+Cross-layer and A/G cross terms vanish in expectation. The prediction terms
+`(1/B) sum mean(p)c^T` and `(1/B) sum mean(pt)c^T` are locally available
+exactly, so subtracting them from `Uhat_A,Uhat_G` gives an unbiased stochastic
+gradient of `J`. BatchNorm changes neither identity: `D` is explicitly the
+summed minibatch derivative and therefore includes its true cross-example
+Jacobian.
+
+This estimator perturbs only the shared mapping that will survive
+amortization. It uses the same two loss queries per direction as the earlier
+estimators. It does not uniformly dominate them for every context covariance
+or batch size, but at the frozen batch-128 synthetic audit its one-direction
+MSE is `0.03381x` the coefficient-then-regress estimator's MSE. The executable
+mean has cosine `0.96928` and norm ratio `1.04888` to the exact A/G target.
+Those mechanics do not establish task utility; a separately frozen gate is
+still required.
+
 ## 2. What is sufficient for a descent step
 
 Flatten all forward parameters into `theta`, let `p = grad L(theta)`, and let