diff options
Diffstat (limited to 'THEORY.md')
| -rw-r--r-- | THEORY.md | 58 |
1 files changed, 55 insertions, 3 deletions
@@ -120,6 +120,57 @@ single-step loss change rather than reporting hidden cosine alone. ## 3. Somato-dendritic innovation as conditional nuisance removal +The regression residual has a stronger interpretation than the linear model +alone suggests. Let `F_h` be the information available to a soma-only +predictor in a neutral period, let `n` denote ordinary apical traffic, and set + +```text +m(h) = E[n | F_h]. +``` + +For every square-integrable soma-measurable predictor `g(h)`, conditional +expectation gives the orthogonality and Pythagorean identities + +```text +E <n - m(h), g(h)> = 0, + +E ||n - g(h)||^2 + = E ||n - m(h)||^2 + E ||m(h) - g(h)||^2. +``` + +Thus `n-m(h)` is the unique minimum-power innovation after subtracting any +function of the available somatic statistic. This is an `L2` projection +statement, not a Gaussian or linear assumption. If task-period apical activity +is `a=s+n` and the neutral traffic law is invariant, neutral fitting yields + +```text +r = a - m(h) = s + n - m(h). +``` + +It removes the largest soma-predictable nuisance component without projecting +out any part of the task-only instruction `s`. The implemented diagonal affine +predictor is the corresponding optimal projection only within each cell's +restricted span `{1,h_i}`; it cannot remove traffic predictable only from +population context or unobserved state. This restriction is consistent with +the failed broad endogenous-traffic gate and prevents the theorem from being +used to claim arbitrary contextual denoising. + +The norm-matched raw control isolates direction from magnitude. For every +positive per-example scalar `alpha`, + +```text +cos(alpha a, -g) = cos(a, -g). +``` + +Choosing `alpha=||r||/||a||` gives raw feedback exactly the innovation's norm +but cannot rotate it toward the descent direction at the same network state. +Across training, rescaling can of course change the subsequent trajectory; +this is why the matched control is still trained end to end rather than being +replaced by the algebraic observation. + +The finite-dimensional special case used for the main experiment writes +neutral traffic as a linear somatic component plus unpredictable noise. + Write apical activity during a task period as ```text @@ -217,6 +268,7 @@ python experiments/verify_theory.py ``` The verifier checks the exact Rademacher MSE across depth, width, and K; the -quadratic finite-sigma bias law; the smooth-descent step threshold; neutral -predictor convergence across `eta_P` and update counts; and the task-fit -teaching-signal absorption predicted above. +quadratic finite-sigma bias law; the smooth-descent step threshold; the +conditional-projection Pythagorean identity and norm-matching direction +invariance; neutral predictor convergence across `eta_P` and update counts; +and the task-fit teaching-signal absorption predicted above. |