path: root/THEORY.md

# SDIL theory and cost audit

This note states the minimum claims that can be proved for the implemented
estimators and innovation rule. It deliberately separates an unbiased neural
teaching direction from a useful parameter update: the completed experiments
show that a high hidden-state cosine can coexist with poor optimization.

## 1. Simultaneous multi-layer perturbation

Consider one example and collect the hidden-state gradients
`g_l = dL/dh_l` for hidden blocks `l=1,...,H`, where `g_l` has dimension
`d_l`. For one simultaneous Rademacher intervention, draw independent
`xi_l in {-1,+1}^{d_l}` and linearize the centered loss difference as

```text
D = sum_j <g_j, xi_j>.
```

The gradient and descent estimators for layer `l` are

```text
g_hat_l = xi_l D,        q_l = -g_hat_l.
```

Independence and `E[xi_l xi_l^T] = I` give

```text
E[g_hat_l] = g_l.
```

Because a Rademacher vector has deterministic squared norm `d_l`, the exact
linearized mean-squared error is

```text
E ||g_hat_l - g_l||^2
  = (d_l - 1) ||g_l||^2
    + d_l sum_{j != l} ||g_j||^2.
```

Averaging `K` independent directions divides this expression by `K`. The first
term is ordinary directional-estimator variance; the second is cross-layer
interference. Thus simultaneous perturbation is unbiased but its relative
variance grows with layer width and with the total gradient energy injected at
other layers. This predicts why K1 can work in an easy/depth-flat regime yet
fail to train a moving context vectorizer on a compositional task.

For comparison, layerwise Rademacher perturbation removes the cross-layer term.
The current layerwise implementation uses Gaussian directions, for which

```text
E ||g_hat_l - g_l||^2 = (d_l + 1) ||g_l||^2 / K.
```

Layerwise targets therefore buy variance reduction with depth-dependent replay
cost; they are not a free scalable replacement for simultaneous calibration.

### Finite perturbation bias

Suppose the third derivative of the loss has operator norm at most `M` in the
perturbation neighborhood, and let `D_total = sum_l d_l`. A centered antithetic
difference has directional error at most

```text
M sigma^2 ||xi||^3 / 6.
```

For simultaneous Rademacher directions this gives the conservative layerwise
bias bound

```text
||E[g_hat_l] - g_l||
  <= (M sigma^2 / 6) sqrt(d_l) D_total^(3/2).
```

The important dependence is `O(sigma^2)`, with a dimension-dependent constant.
Reducing K removes variance but not this bias; increasing width/depth without a
sigma audit is therefore unsafe.

### Perturbing the representable convolutional subspace

The failed ResNet-20 run exposed a separate projection problem.  Its
translation-shared feedback field for example `i`, channel `c`, and spatial
site `s` is

```text
p_ics = b_ic + t_ics v_ic,       t_ics = tanh(h_ics).
```

The legacy estimator perturbs every `h_ics` independently, constructs a full
unit target, and only then averages its delta-rule update into the shared
`b/v` parameterization.  At K1, almost all sampled energy can lie outside this
two-dimensional per-channel subspace.  SDIL-v2 instead draws independent
Rademacher coefficients `z^b_ic,z^v_ic` and intervenes along

```text
xi_ics = (z^b_ic + t_ics z^v_ic) / sqrt(2).
```

Let `q_ics=-dL/dh_ics`, let `S` be the number of spatial sites, and let `D` be
the centered directional loss derivative.  Then

```text
qhat^b_ic = -sqrt(2) D z^b_ic / S,
qhat^v_ic = -sqrt(2) D z^v_ic / S
```

are unbiased for the two negative-gradient moments

```text
qbar^b_ic = mean_s q_ics,
qbar^v_ic = mean_s (q_ics t_ics).
```

All other examples, channels, and layers contribute zero-mean cross terms.
For BatchNorm, `D` is the derivative of the summed minibatch loss, so the same
identity targets each example's contribution including the true cross-example
Jacobian coupling.

This does not mistake moments for expansion coefficients.  The predicted
moments of `p=b+t v` are

```text
pbar^b = b + mean(t) v,
pbar^v = mean(t) b + mean(t^2) v.
```

Consequently updating the two vectorizer pathways with
`qhat^b-pbar^b` and `qhat^v-pbar^v` is exactly the expected gradient of the
original full-field squared prediction error.  It uses the same two causal
loss queries per direction as unit perturbation, but avoids estimating
unrepresentable spatial components.  It remains a variance reduction, not a
guarantee of useful feedback or stable end-to-end optimization; those require
a separately frozen empirical gate.

### Perturbing the vectorizer parameter subspace

The channel-moment estimator still samples one independent base/gate
coefficient for every example and channel and only afterward regresses those
noisy coefficients on the output context.  The quantity the local delta rule
actually needs is much smaller.  For

```text
p_ics = (A c_i)_c + t_ics (G c_i)_c
```

define the normalized prediction objective

```text
J(A,G) = (1 / 2B) sum_i mean_s ||p_i - q_i||^2,
q_i = -d ell_i / dh_i.
```

Its causal target terms are matrices, not per-example coefficient fields:

```text
U_A = (1/B) sum_i mean_s(q_ics) c_i^T,
U_G = (1/B) sum_i mean_s(q_ics t_ics) c_i^T.
```

Draw independent Rademacher matrices `Xi_A,Xi_G` with the same shapes as
`A,G`, independently across layers, and intervene along

```text
d_ics = ((Xi_A c_i)_c + t_ics (Xi_G c_i)_c) / sqrt(2).
```

Let `D` be the centered derivative of the summed minibatch loss under all
layer interventions. Since `D=-sum_ics q_ics d_ics`, independence gives

```text
Uhat_A = -sqrt(2) D Xi_A / (B S),     E[Uhat_A] = U_A,
Uhat_G = -sqrt(2) D Xi_G / (B S),     E[Uhat_G] = U_G.
```

Cross-layer and A/G cross terms vanish in expectation. The prediction terms
`(1/B) sum mean(p)c^T` and `(1/B) sum mean(pt)c^T` are locally available
exactly, so subtracting them from `Uhat_A,Uhat_G` gives an unbiased stochastic
gradient of `J`. BatchNorm changes neither identity: `D` is explicitly the
summed minibatch derivative and therefore includes its true cross-example
Jacobian.

This estimator perturbs only the shared mapping that will survive
amortization. It uses the same two loss queries per direction as the earlier
estimators. It does not uniformly dominate them for every context covariance
or batch size, but at the frozen batch-128 synthetic audit its one-direction
MSE is `0.03381x` the coefficient-then-regress estimator's MSE. The executable
mean has cosine `0.96928` and norm ratio `1.04888` to the exact A/G target.
Those mechanics do not establish task utility; a separately frozen gate is
still required.

### Perturbing the hierarchical feedback-parameter subspace

The hierarchical oracle shows that an early field needs the spatial child
teaching map rather than only the output-error vector. For a residual-DAG edge
`e`, let `s_e` be the locally gated and BatchNorm-transformed child field and
let the feedback contribution at its parent be

```text
p_e = conv_transpose(s_e, Q_e).
```

The complete predicted parent field `p` also includes the exact
parameter-free shortcut contribution. Draw an independent Rademacher tensor
`Xi_e` with the shape of every feedback convolution and intervene at each
edge's parent along

```text
d_e = conv_transpose(s_e, Xi_e).
```

The readout-feedback matrix is treated identically, with
`d_out=(c Xi_out^T)/S` at the final spatial map. All edge fields are injected
in one antithetic pair. If `D` is the derivative of the summed minibatch loss,
independence across parameter tensors gives

```text
E[-D Xi_e / (B S_parent)]
  = corr(q_parent, s_e) / (B S_parent),
q_parent = -d ell / d h_parent.
```

All other edges contribute zero-mean cross terms. The locally available
prediction moment is

```text
corr(p_parent, s_e) / (B S_parent),
```

where `p_parent` includes both convolutional and shortcut feedback. Therefore

```text
Delta Q_e = eta_A [target_moment_e - prediction_moment_e]
```

is an unbiased stochastic delta rule for the normalized squared parent-field
error. It uses two task-loss queries per direction, no copied forward tensor,
and no reverse differentiation. Unlike V2/V3, its perturbation family already
contains the spatial child teaching context. Unlike weight mirroring, its
target is task-causal credit rather than reconstruction of the forward
convolution. Recursive feedback calibration and feedback convolutions are not
claimed as novel by themselves; this is an engineering prerequisite for a
hierarchical innovation model and requires direct HFA, learned-FA/mirroring,
and BurstCCN comparisons.

The executable audit checks the complete BatchNorm-coupled antithetic JVP to
`6.2e-10` relative error. Under an audit-only symmetric feedback copy, the
predicted Q/R moments equal the exact causal delta-rule targets to `2.3e-16`
relative error. Actual training never performs that copy. These identities
establish mechanics only; a frozen-forward causal-capture gate must pass before
any task-scale endpoint is opened. That empirical V4 gate subsequently failed:
the stable rate left early alignment near zero, while larger rates caused a
58.5x feedback norm ratio or nonfinite state. The unbiased identity therefore
does not imply usable finite-budget signal in the 267,904-dimensional joint
parameter space.

### Local response mirroring as an inherited strong baseline

V4's failure motivates changing the observation, not merely its rate. The
weight-mirror identity of Akrout et al. starts from white local parent noise
`z` and its child response `u=Wz`:

```text
E[z u^T] = sigma^2 W^T.
```

For convolutions, the corresponding local weight correlation divided by the
number of valid spatial probe pairs and `sigma^2` is an unbiased estimator of
the forward kernel. The normalized response-mirror baseline uses a
bias-blocked pre-BatchNorm observation phase and updates

```text
What_e = corr(z_e, u_e) / (count_e sigma^2),
Q_e <- (1-eta_M) Q_e + eta_M What_e.
```

The readout is identical, with the sign reversed to match this repository's
negative-gradient teaching convention. Observation generation uses the
ordinary local forward synapses; the Q/R update API receives only probe and
response activities. An executable independence audit changes every forward
parameter after observation and verifies that the feedback update is bitwise
unchanged.

This is a stabilized system-identification variant of
[weight mirroring](https://proceedings.neurips.cc/paper_files/paper/2019/file/f387624df552cea2f369918c5e1e12bc-Paper.pdf),
not an SDIL contribution. The original Hebbian-plus-decay rule can be highly
metaparameter-sensitive, as analyzed by
[Kunin et al.](https://proceedings.mlr.press/v119/kunin20a/kunin20a.pdf);
normalizing the local sufficient statistic makes the estimator's scale
explicit. At the deterministic smoke setting, one 16-probe observation has
mean/minimum feedback-forward cosine `0.9887/0.9661`, norm ratios
`[0.9537,1.0444]`, and zero forward-parameter access error in the update.
These are mechanics, not task evidence. A frozen comparator gate is required.

The frozen WM short task gate later exposed a finite-sample defect of the
estimate-then-average rule: each new `What` remains noisy even when `Q=W`, so
an exponential tracker has a nonzero stationary error. Residual response
mirroring (RRM) makes a substantive rule change. Given the same local probe
and observed child response, it predicts the child response through Q and uses
only the residual:

```text
u       = conv(z, W),
uhat    = conv(z, Q),
Delta Q = eta_M corr(z, u-uhat) / (count sigma^2).
```

In expectation this is the same contraction toward W, but when Q exactly
matches W the update is zero for every individual probe rather than only in
expectation. The readout uses the identical signed response residual. An
executable audit measures fixed-point update RMS `2.75e-18`, response-residual
fraction `3.08e-17`, and zero update dependence on forward parameters after
the observations are fixed. RRM needs one extra local Q prediction convolution
per mirror event, which is charged separately. It remains inherited local
weight estimation, not SDIL novelty.

### Reciprocal Kolen--Pollack dynamics as the stable inherited substrate

The modified Kolen--Pollack comparator does not estimate a moving weight from
intermittent probes. Corresponding forward and reciprocal synapses independently
form the same local activity correlation `C_t` on every task update. With
matched momentum `mu`, decay `lambda`, and rate `eta`, their difference state
obeys

```text
M_(t+1) = mu M_t - lambda D_t,
D_(t+1) = D_t + eta M_(t+1),
D_t = Q_t - W_t,       M_t = mQ_t - mW_t.
```

The task correlation cancels exactly; with no momentum this reduces to
`D_(t+1)=(1-eta lambda)D_t`. Exact symmetry is an invariant for the matched
momentum system, and the executable audit obtains zero difference after two
updates. The readout follows the same identity after applying this codebase's
feedback sign convention `R=-W_out^T`. Each reciprocal correlation is
recomputed from its own local activity pair; neither path reads the other
weight or update.

Mixed apical traffic does not change that invariant. If `u_l` is raw,
norm-matched raw, or innovation, both corresponding synapses consume the same
local parent activity and `u_l`, hence independently form equal `C_t`. The
traffic/predictor mechanism can therefore be ablated without silently changing
the credit-substrate tracking rule. Reciprocal KP and its stability are prior
art; only a load-bearing neutral innovation can contribute SDIL evidence.

## 2. What is sufficient for a descent step

Flatten all forward parameters into `theta`, let `p = grad L(theta)`, and let
`v` be the complete local update before multiplying by the learning rate:

```text
theta_next = theta + eta v.
```

If the loss is `beta`-smooth and

```text
c = <-p, v> / (||p|| ||v||) > 0,
```

then the descent lemma gives

```text
L(theta_next) - L(theta)
 <= -eta c ||p|| ||v|| + (beta eta^2 / 2) ||v||^2.
```

Consequently the update is guaranteed to descend only when

```text
0 < eta < 2 c ||p|| / (beta ||v||).
```

This separates three quantities that a neural-state cosine conflates:

1. direction (`c`);
2. gain (`||v|| / ||p||`);
3. curvature (`beta`).

For a local layer, multiplying a hidden teaching vector by the activation gain
and presynaptic activity maps it into parameter space. A positive
`cos(r_l, -g_l)` does not by itself imply a positive cosine after aggregating
examples, parameters, and layers, and even perfect parameter alignment can
increase loss if its gain makes the step exceed the bound. This is why future
diagnostics must include parameter-update cosine, norm ratio, and realized
single-step loss change rather than reporting hidden cosine alone.

## 3. Somato-dendritic innovation as conditional nuisance removal

The regression residual has a stronger interpretation than the linear model
alone suggests. Let `F_h` be the information available to a soma-only
predictor in a neutral period, let `n` denote ordinary apical traffic, and set

```text
m(h) = E[n | F_h].
```

For every square-integrable soma-measurable predictor `g(h)`, conditional
expectation gives the orthogonality and Pythagorean identities

```text
E <n - m(h), g(h)> = 0,

E ||n - g(h)||^2
  = E ||n - m(h)||^2 + E ||m(h) - g(h)||^2.
```

Thus `n-m(h)` is the unique minimum-power innovation after subtracting any
function of the available somatic statistic. This is an `L2` projection
statement, not a Gaussian or linear assumption. If task-period apical activity
is `a=s+n` and the neutral traffic law is invariant, neutral fitting yields

```text
r = a - m(h) = s + n - m(h).
```

It removes the largest soma-predictable nuisance component without projecting
out any part of the task-only instruction `s`. The implemented diagonal affine
predictor is the corresponding optimal projection only within each cell's
restricted span `{1,h_i}`; it cannot remove traffic predictable only from
population context or unobserved state. This restriction is consistent with
the failed broad endogenous-traffic gate and prevents the theorem from being
used to claim arbitrary contextual denoising.

The norm-matched raw control isolates direction from magnitude. For every
positive per-example scalar `alpha`,

```text
cos(alpha a, -g) = cos(a, -g).
```

Choosing `alpha=||r||/||a||` gives raw feedback exactly the innovation's norm
but cannot rotate it toward the descent direction at the same network state.
Across training, rescaling can of course change the subsequent trajectory;
this is why the matched control is still trained end to end rather than being
replaced by the algebraic observation.

The finite-dimensional special case used for the main experiment writes
neutral traffic as a linear somatic component plus unpredictable noise.

Write apical activity during a task period as

```text
a = s + M h + epsilon,
```

where `s` is the instructional component, `M h` is ordinary traffic predictable
from soma, and `epsilon` is unpredictable noise. In neutral periods `s=0`.
Assume `Sigma_h = E[h h^T]` is nonsingular and `E[epsilon h^T]=0`. Population
least squares on neutral periods yields

```text
P_neutral = E[a h^T | neutral] Sigma_h^(-1) = M.
```

The task-period innovation is then

```text
r_neutral = a - P_neutral h = s + epsilon.
```

If `s`, `M h`, and `epsilon` are mutually uncorrelated, a squared-cosine/SNR
proxy improves from

```text
raw:       S / (S + T + E)
innovation:S / (S + E),
```

where `S`, `T`, and `E` are their expected squared norms. The gain is strict
whenever predictable traffic has nonzero power.

Task-period fitting has a different estimand. Let

```text
C = E[s h^T] Sigma_h^(-1).
```

Then `P_task = M + C` and

```text
r_task = s - C h + epsilon.
```

It removes the soma-predictable projection of the teaching signal itself. This
proves the identification role of neutral periods and also defines the failure
boundary: residualization cannot remove contextual traffic that remains
unpredictable from the chosen per-cell soma statistic, and it can erase credit
when the predictor is fit on task periods.

For the diagonal normalized-LMS predictor used in the code, an ideal noiseless
cell with neutral relation `a=m h` follows

```text
p_(t+1) = p_t + eta_P (m - p_t),
traffic residual power fraction = (1 - eta_P)^(2t).
```

The relevant timescale is therefore the product of learning rate and number of
neutral updates. Too few neutral updates leave traffic; task-period updates
converge to the wrong coefficient regardless of speed.

### Incomplete residualization can create multiplicative local instability

Residual power alone is not a stability certificate. Consider a linearized
local population `h=W x` with input covariance `C=E[x x^T]`. If the true
soma-proportional traffic coefficient is `M` and the neutral predictor is `P`,
write the remaining coupling as `D=M-P`. The innovation used for the local
update is

```text
r = s + D h = s + D W x.
```

Ignoring nonlinear gates for this local calculation, the expected forward
update with decay is

```text
E[Delta W] = eta (S + D W C - lambda W),
S = E[s x^T].
```

Thus predictor error does not enter as additive noise. It multiplies the
current forward weights through `D W C`; a positive mode is a Hebbian feedback
loop. In vector form its homogeneous operator is

```text
vec(W_(t+1))
  = [I + eta (C^T kron D - lambda I)] vec(W_t).
```

The implemented momentum dynamics make the same point sharply. Along a scalar
joint eigenmode let `k=d c-lambda`, where `d` and `c>=0` are eigenvalues of
`D` and `C`. With momentum `mu`, the homogeneous state follows

```text
[w_(t+1)]   [1 + eta k, eta mu] [w_t]
[m_(t+1)] = [k,          mu   ] [m_t].
```

The determinant is `mu`. Evaluating its characteristic polynomial at one
gives `p(1)=-eta k`; therefore every `k>0` produces a real eigenvalue above
one. The full second-order Jury conditions give the strict stability window

```text
|mu| < 1,
-2 (1 + mu) / eta < k < 0.
```

The upper boundary excludes positive Hebbian feedback; the lower boundary
excludes an over-strong negative mode that alternates and grows. Weight decay
or predictor overestimation must therefore control both sign and magnitude,
not merely make `k` nonpositive. Consequently a small aggregate residual RMS
can coexist with an unstable worst-direction coefficient. Deep state
dependence can amplify the same mode across layers or increase the local
covariance until an initially stable negative `k` crosses the lower boundary.

This calculation explains the frozen MT-1 failure boundary without changing
its result. The training-only audit finds the first active nonfinite tensors
at the stem forward weight and momentum: step 23 for raw, 69 for norm-matched
raw, and 74 for innovation. Residualization and norm matching delay the
positive-feedback mode but do not stabilize it at the frozen four-to-one
traffic setting. A scalable version needs an operator-level dissipativity or
gain-control mechanism; the pretraining residual-RMS threshold is
insufficient by itself.

The subsequent frozen S0 training-prefix screen tests a fixed one-sided
predictor margin and finds no eligible value in `{0.001,0.003,0.01,0.03}`.
Every candidate has nonpositive measured neutral residual-soma slope, yet the
smaller margins overflow BatchNorm state and the larger margins show enormous
transient loss and parameter growth. This is consistent with the two-sided
condition above: sign control is necessary but does not bound the evolving
gain `d c` or nonnormal deep-network transients.

The post-S0 dynamic controller instead separates a slow neutral predictor from
a fast instruction-off projection. On the current minibatch, let its neutral
residual be `e0=a0-P h-b`. Each cell forms

```text
q_t = Cov_t(e0, h) / Var_t(h),
e_perp = (e0 - mean_t(e0)) - q_t (h - mean_t(h)).
```

Only `s+e_perp` enters plasticity. The fit for `q_t` never observes `s`, so it
cannot regress away a task instruction merely because that instruction is
correlated with soma. For the diagonal affine traffic intervention,
`e0=D h+c` and hence `e_perp=0` in exact arithmetic at every task state. The
effective homogeneous coefficient is then `k=-lambda`, which satisfies both
Jury bounds for the frozen positive learning rate, momentum below one, and
small positive decay. Unlike a fixed negative margin, nulling `D` is invariant
to changes in the nonnegative input-covariance eigenvalue `c`.

This is an empirical local certificate for the intervention being tested, not
a global stability theorem for a nonlinear ResNet. Finite-batch regression,
roundoff, cross-layer nonnormality, and traffic outside the diagonal affine
family remain possible failure modes. The separately frozen D1 training-prefix
gate in `DYNAMIC_INNOVATION.md` therefore requires both the measured local
certificate and bounded loss, parameter, optimizer, and BatchNorm trajectories
before any validation endpoint can open.

### Intermittent feedback tracking can hide behind a final cosine

An idealized mirror event every `k` task updates uses

```text
Q_m^+ = Q_m + eta_M (W_m - Q_m).
```

Let `E_m=Q_m-W_m` immediately before that event and let
`U_m=W_(m+1)-W_m` be the sum of the following `k` forward-weight changes.
The tracking error obeys the exact recurrence

```text
E_(m+1) = (1 - eta_M) E_m - U_m.
```

If every task update has norm at most `d`, then `||U_m|| <= k d` and

```text
limsup_m ||E_m|| <= k d / eta_M.
```

The bound is attained in the scalar or constant-collinear case. Thus reducing
the mirror cadence or increasing its learning rate only controls lag relative
to how fast the task weights move; a high feedback cosine is not automatic.
For the frozen RRM setting `k=16, eta_M=0.1`, the ideal worst-case lag scale is
`160` single-task-update norms before stochastic finite-probe error is added.

Conversely, after the task updates stop (`U_m=0`), the endpoint error decays as
`(1-eta_M)^m`. A learning-rate drop or quiet tail can therefore produce an
excellent final cosine even when the feedback was badly out of date during
the loss-producing part of training. Endpoint alignment must be reported with
the alignment trajectory and task loss. This is a standard tracking fact used
to audit the mirror baseline, not an SDIL novelty claim.

### Why instantaneous gradient feedback cannot reproduce the BCI derivative sign

The failed oral-B model calibrated its apical vectorizer to the instantaneous
negative loss gradient. Let `e_t>0` be the target-minus-cursor error during an
active trial and let `s_i` be the signed causal role of cell `i` (`s_i>0` for
P+, `s_i<0` for P-). With squared loss, the perfect calibration target is

```text
r_i(t) = e_t s_i.
```

Consequently the P+ minus P- residual contrast is a positive constant times
`e_t`. Whenever current error is smaller in error-reduction epochs than in
error-increase epochs, the Harnett separability statistic

```text
0.5 * [E(P+ - P- | error decreases)
       - E(P+ - P- | error increases)]
```

is negative. Adding error velocity as an input feature does not fix this:
regression against `e_t s_i` still assigns the causal coefficient to error
magnitude and has no reason to assign it to velocity. The complete negative
36-run screen is therefore a structural falsification of that target, not a
learning-rate failure.

A derivative-faithful candidate separates causal-role identification from the
temporal performance modulator. With Rademacher perturbations `xi`, observe
only the scalar cursor under antithetic state interventions:

```text
z+ = s^T (h + sigma xi),       z- = s^T (h - sigma xi),
q_i = ((z+ - z-) / (2 sigma)) xi_i.
```

Then `E[q_i]=s_i`; no task loss, gradient, transpose weight, or reverse graph
is used. Define the within-episode performance innovation

```text
delta_t = |e_(t-1)| - |e_t|,
r_i(t) = delta_t * m_i,        m_i approximately s_i.
```

Now the same separability statistic is positive by construction when both
improving and worsening epochs occur. More importantly, this is not merely a
metric relabelling: `delta_t` is the scalar outcome that modulates the local
eligibility update, while sparse causal perturbations amortize the
neuron-specific role vector `m`. Whether that three-factor temporal rule
actually learns, preserves residual decorrelation, and survives a plasticity
lesion remains an empirical gate. Direct online control is not implied; in
fact applying `r` as an immediate state correction would reinforce both good
and bad velocity signs and is therefore excluded from the first recovery.

## 4. Hardware-independent complexity

Let `H` be the number of hidden layers, `K` the perturbation directions per
event, `e` the number of ordinary minibatches between events, `B` the batch
size, and `F_l` the affine work of layer `l`. Define `F=sum_l F_l` and suffix
work `S_l=sum_{j>l} F_j`.

| method | activity phases / passes | transport | scalar task-loss observations | extra affine work | main transient memory |
|:--|:--|:--|:--|:--|:--|
| BP | forward + reverse | exact transposed Jacobians | ordinary supervised loss | about one reverse pass | saved activations / graph |
| FA | forward + reverse-like serial feedback | fixed random feedback | ordinary loss | `O(F)` feedback work | activations + feedback states |
| convolutional hierarchical FA | forward + residual-DAG feedback | independent random 3x3 feedback tensors; exact parameter-free shortcut graph | ordinary loss | one feedback convolution per non-stem forward convolution (`0.988x` forward MACs at ResNet-20, plus local correlations) | hidden feedback maps + forward caches |
| normalized / residual response mirror | hierarchical task pass plus intermittent local probe-response events | learned Q/R from local parent-child observations; no task-loss query | ordinary loss only | per mirror event: forward response + local correlation; RRM also predicts the response through Q | task caches plus one probe/response set |
| reciprocal Kolen--Pollack | hierarchical task pass; forward and reciprocal correlations every update | separate plastic Q/R; equal local activity products and matched optimizer/decay | ordinary loss only | forward local correlation + feedback propagation + a separately recomputed reciprocal correlation (`1.326x` matched BP MACs in KP-1) | task caches plus feedback fields and reciprocal optimizer state |
| KP + somato-dendritic innovation | KP plus neutral predictor warmup/update | same reciprocal Q/R; per-unit affine neutral predictor | ordinary loss only | KP affine work plus explicitly reported traffic/predictor elementwise operations and two charged diagnostic-prefix forwards | KP state plus raw/innovation fields and `2U` predictor parameters |
| DFA | forward + direct feedback | fixed random output maps | ordinary loss | `O(sum_l d_l d_out)` | activations + feedback states |
| learned NP, simultaneous | ordinary forward; one duplicate clean plus `2K` noisy full forwards every `e` batches | learned direct maps; no weight transport | `2KB/e` per ordinary batch | `(1+2K)F/e` | up to `2KB` noisy examples in the vectorized implementation |
| learned NP, layerwise | ordinary forward; clean baseline plus antithetic suffix replays | learned direct maps | `(1+2KH)B/e` | `(F + 2K sum_l S_l)/e`, quadratic in depth for balanced layers | serial suffix replay, no `2K` batch expansion |
| direct NP | layerwise or simultaneous estimator every update | none beyond causal intervention | same formulas with `e=1` | same formulas with `e=1` | estimator dependent |
| canonical EP | free relaxation then nudged relaxation | symmetric energy weights | label injected in nudged phase | `T_free + T_nudged` recurrent steps | persistent/free and nudged states |
| Dual Prop `DP^T` VGG | one inference phase, 17 forward-only layer passes in the author CIFAR protocol | dyadic forward/top-down states | ordinary target loss | 17 layer passes before update | two states per dyadic unit |
| BurstCCN | one burst/event-probability phase with plastic W/Y/Q pathways | separate plastic feedback pathways | ordinary supervised target | architecture-dependent local W/Y/Q updates | event and burst states plus feedback weights |

The project ledger reports the exact realized loss observations,
forward-equivalent examples, peak allocated memory, and wall time. Asymptotic
labels never replace those measurements. In particular, the frozen direct-NP
C2 diagnosis costs `68.4x` ordinary work, whereas the frozen CIFAR d20 K1/e4
protocol uses 16x fewer logical queries, 11x less calibration work, and 5.3x
less total work than K16/e4 while retaining its gain over DFA.

## 5. Executable checks

Run:

```bash
python experiments/verify_theory.py
```

The verifier checks the exact Rademacher MSE across depth, width, and K; the
quadratic finite-sigma bias law; the smooth-descent step threshold; the
conditional-projection Pythagorean identity and norm-matching direction
invariance; neutral predictor convergence across `eta_P` and update counts;
the multiplicative residual-coupling operator and its unstable positive mode;
the task-fit teaching-signal absorption predicted above; and the BCI
instantaneous-gradient sign conflict versus the temporal-difference role
factorization.