diff options
Diffstat (limited to 'THEORY.md')
| -rw-r--r-- | THEORY.md | 72 |
1 files changed, 72 insertions, 0 deletions
@@ -499,6 +499,78 @@ The relevant timescale is therefore the product of learning rate and number of neutral updates. Too few neutral updates leave traffic; task-period updates converge to the wrong coefficient regardless of speed. +### State-dependent component bias accumulates with system size + +The physical-network setting makes the nuisance-removal statement concrete. +For component `e`, let `z_e` denote locally observable state, `g_e` the ideal +credit signal, and let the measured local signal be + +```text +a_e = g_e + b_e(z_e) + epsilon_e, +``` + +where `b_e(z_e)` is a repeatable component imperfection and `epsilon_e` is the +remaining zero-mean variation. A one-time constant calibration subtracts +`c_e=E[b_e]`. The best predictor from the local information `F_e` is +`p_e=E[b_e | F_e]`. Conditional projection gives + +```text +E[(b_e-c_e)^2] - E[(b_e-p_e)^2] + = E[(p_e-c_e)^2] >= 0. +``` + +Thus constant calibration is optimal only when the predictable imperfection is +constant over the component's operating states. SDIL estimates `p_e` online +from neutral observations and subtracts it locally; it does not require the +simulator's imperfection parameters or a component-by-component oracle table. +Its remaining error includes predictor approximation, tracking error, and the +unpredictable part `epsilon_e`. + +For `E` components, the squared norm of the residual bias is exactly the sum +of its component powers: + +```text +E[||delta||^2] = sum_e E[delta_e^2]. +``` + +If the average residual power per component remains bounded away from zero, +the total residual power grows linearly in component count. Correlation between +components can change particular projected modes, but it does not remove this +sum-of-squares accumulation. Reducing the per-component residual therefore +reduces the coefficient of size-dependent error growth; it need not make a +large physical network perfect. + +The link from residual bias to optimization error can be stated exactly for a +local quadratic model. Let + +```text +F(k) = 1/2 (k-k_star)^T H (k-k_star), +implemented update = -grad F(k) + delta. +``` + +For constant `delta` in the range of positive-semidefinite `H`, the displaced +fixed point and its excess objective are + +```text +k_infinity = k_star + H^+ delta, +F(k_infinity)-F(k_star) = 1/2 delta^T H^+ delta. +``` + +On the positive-curvature subspace this implies + +```text +||delta||^2/(2 lambda_max) + <= F(k_infinity)-F(k_star) + <= ||delta||^2/(2 lambda_min). +``` + +A component of `delta` in the nullspace of `H` causes drift rather than a +displaced optimum. These statements explain why persistent bias can produce a +size-dependent floor and why reducing residual power can reduce that floor. +They do not by themselves determine classification error: the CLLN ladder +tests the downstream effect using held-out logic tasks, while residual-bias +measurements remain diagnostic. + ### Incomplete residualization can create multiplicative local instability Residual power alone is not a stability certificate. Consider a linearized |