summaryrefslogtreecommitdiff
path: root/THEORY.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 13:33:33 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 13:33:33 -0500
commit0db651ac83608ea8f5b94950a391da22f076a142 (patch)
treeaae335ea16eb1b2f8981ba1217b2f91cdc2ae9b9 /THEORY.md
parentc27e0824a41e6c54dd9079f17c20c306c5f2e9c9 (diff)
baseline: add normalized local response mirroring
Diffstat (limited to 'THEORY.md')
-rw-r--r--THEORY.md38
1 files changed, 38 insertions, 0 deletions
diff --git a/THEORY.md b/THEORY.md
index 14c585f..7038969 100644
--- a/THEORY.md
+++ b/THEORY.md
@@ -253,6 +253,44 @@ the stable rate left early alignment near zero, while larger rates caused a
does not imply usable finite-budget signal in the 267,904-dimensional joint
parameter space.
+### Local response mirroring as an inherited strong baseline
+
+V4's failure motivates changing the observation, not merely its rate. The
+weight-mirror identity of Akrout et al. starts from white local parent noise
+`z` and its child response `u=Wz`:
+
+```text
+E[z u^T] = sigma^2 W^T.
+```
+
+For convolutions, the corresponding local weight correlation divided by the
+number of valid spatial probe pairs and `sigma^2` is an unbiased estimator of
+the forward kernel. The normalized response-mirror baseline uses a
+bias-blocked pre-BatchNorm observation phase and updates
+
+```text
+What_e = corr(z_e, u_e) / (count_e sigma^2),
+Q_e <- (1-eta_M) Q_e + eta_M What_e.
+```
+
+The readout is identical, with the sign reversed to match this repository's
+negative-gradient teaching convention. Observation generation uses the
+ordinary local forward synapses; the Q/R update API receives only probe and
+response activities. An executable independence audit changes every forward
+parameter after observation and verifies that the feedback update is bitwise
+unchanged.
+
+This is a stabilized system-identification variant of
+[weight mirroring](https://proceedings.neurips.cc/paper_files/paper/2019/file/f387624df552cea2f369918c5e1e12bc-Paper.pdf),
+not an SDIL contribution. The original Hebbian-plus-decay rule can be highly
+metaparameter-sensitive, as analyzed by
+[Kunin et al.](https://proceedings.mlr.press/v119/kunin20a/kunin20a.pdf);
+normalizing the local sufficient statistic makes the estimator's scale
+explicit. At the deterministic smoke setting, one 16-probe observation has
+mean/minimum feedback-forward cosine `0.9887/0.9661`, norm ratios
+`[0.9537,1.0444]`, and zero forward-parameter access error in the update.
+These are mechanics, not task evidence. A frozen comparator gate is required.
+
## 2. What is sufficient for a descent step
Flatten all forward parameters into `theta`, let `p = grad L(theta)`, and let