From 26c257fa528118671a5914f9f9796b24d451839f Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Wed, 22 Jul 2026 16:56:13 -0500 Subject: theory: bound both sides of residual stability --- THEORY.md | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-) (limited to 'THEORY.md') diff --git a/THEORY.md b/THEORY.md index 8299c75..f659d0f 100644 --- a/THEORY.md +++ b/THEORY.md @@ -539,11 +539,20 @@ joint eigenmode let `k=d c-lambda`, where `d` and `c>=0` are eigenvalues of The determinant is `mu`. Evaluating its characteristic polynomial at one gives `p(1)=-eta k`; therefore every `k>0` produces a real eigenvalue above -one. Weight decay stabilizes the mode only if it makes the effective `k` -nonpositive, with the usual additional discrete-time step-size constraints. -Consequently a small aggregate residual RMS can coexist with an unstable -worst-direction coefficient. Deep state dependence can amplify the same mode -across layers before a slow neutral predictor catches it. +one. The full second-order Jury conditions give the strict stability window + +```text +|mu| < 1, +-2 (1 + mu) / eta < k < 0. +``` + +The upper boundary excludes positive Hebbian feedback; the lower boundary +excludes an over-strong negative mode that alternates and grows. Weight decay +or predictor overestimation must therefore control both sign and magnitude, +not merely make `k` nonpositive. Consequently a small aggregate residual RMS +can coexist with an unstable worst-direction coefficient. Deep state +dependence can amplify the same mode across layers or increase the local +covariance until an initially stable negative `k` crosses the lower boundary. This calculation explains the frozen MT-1 failure boundary without changing its result. The training-only audit finds the first active nonfinite tensors @@ -554,6 +563,14 @@ traffic setting. A scalable version needs an operator-level dissipativity or gain-control mechanism; the pretraining residual-RMS threshold is insufficient by itself. +The subsequent frozen S0 training-prefix screen tests a fixed one-sided +predictor margin and finds no eligible value in `{0.001,0.003,0.01,0.03}`. +Every candidate has nonpositive measured neutral residual-soma slope, yet the +smaller margins overflow BatchNorm state and the larger margins show enormous +transient loss and parameter growth. This is consistent with the two-sided +condition above: sign control is necessary but does not bound the evolving +gain `d c` or nonnormal deep-network transients. + ### Intermittent feedback tracking can hide behind a final cosine An idealized mirror event every `k` task updates uses -- cgit v1.2.3