From 0db651ac83608ea8f5b94950a391da22f076a142 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Wed, 22 Jul 2026 13:33:33 -0500 Subject: baseline: add normalized local response mirroring --- THEORY.md | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) (limited to 'THEORY.md') diff --git a/THEORY.md b/THEORY.md index 14c585f..7038969 100644 --- a/THEORY.md +++ b/THEORY.md @@ -253,6 +253,44 @@ the stable rate left early alignment near zero, while larger rates caused a does not imply usable finite-budget signal in the 267,904-dimensional joint parameter space. +### Local response mirroring as an inherited strong baseline + +V4's failure motivates changing the observation, not merely its rate. The +weight-mirror identity of Akrout et al. starts from white local parent noise +`z` and its child response `u=Wz`: + +```text +E[z u^T] = sigma^2 W^T. +``` + +For convolutions, the corresponding local weight correlation divided by the +number of valid spatial probe pairs and `sigma^2` is an unbiased estimator of +the forward kernel. The normalized response-mirror baseline uses a +bias-blocked pre-BatchNorm observation phase and updates + +```text +What_e = corr(z_e, u_e) / (count_e sigma^2), +Q_e <- (1-eta_M) Q_e + eta_M What_e. +``` + +The readout is identical, with the sign reversed to match this repository's +negative-gradient teaching convention. Observation generation uses the +ordinary local forward synapses; the Q/R update API receives only probe and +response activities. An executable independence audit changes every forward +parameter after observation and verifies that the feedback update is bitwise +unchanged. + +This is a stabilized system-identification variant of +[weight mirroring](https://proceedings.neurips.cc/paper_files/paper/2019/file/f387624df552cea2f369918c5e1e12bc-Paper.pdf), +not an SDIL contribution. The original Hebbian-plus-decay rule can be highly +metaparameter-sensitive, as analyzed by +[Kunin et al.](https://proceedings.mlr.press/v119/kunin20a/kunin20a.pdf); +normalizing the local sufficient statistic makes the estimator's scale +explicit. At the deterministic smoke setting, one 16-probe observation has +mean/minimum feedback-forward cosine `0.9887/0.9661`, norm ratios +`[0.9537,1.0444]`, and zero forward-parameter access error in the update. +These are mechanics, not task evidence. A frozen comparator gate is required. + ## 2. What is sufficient for a descent step Flatten all forward parameters into `theta`, let `p = grad L(theta)`, and let -- cgit v1.2.3