diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 16:56:13 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 16:56:13 -0500 |
| commit | 26c257fa528118671a5914f9f9796b24d451839f (patch) | |
| tree | 8b25903e21b6e51cf4885f049637d115ab28c059 /THEORY.md | |
| parent | 8d28fd9f9e85f5e0715b3be2337b7843fc4c943f (diff) | |
theory: bound both sides of residual stability
Diffstat (limited to 'THEORY.md')
| -rw-r--r-- | THEORY.md | 27 |
1 files changed, 22 insertions, 5 deletions
@@ -539,11 +539,20 @@ joint eigenmode let `k=d c-lambda`, where `d` and `c>=0` are eigenvalues of The determinant is `mu`. Evaluating its characteristic polynomial at one gives `p(1)=-eta k`; therefore every `k>0` produces a real eigenvalue above -one. Weight decay stabilizes the mode only if it makes the effective `k` -nonpositive, with the usual additional discrete-time step-size constraints. -Consequently a small aggregate residual RMS can coexist with an unstable -worst-direction coefficient. Deep state dependence can amplify the same mode -across layers before a slow neutral predictor catches it. +one. The full second-order Jury conditions give the strict stability window + +```text +|mu| < 1, +-2 (1 + mu) / eta < k < 0. +``` + +The upper boundary excludes positive Hebbian feedback; the lower boundary +excludes an over-strong negative mode that alternates and grows. Weight decay +or predictor overestimation must therefore control both sign and magnitude, +not merely make `k` nonpositive. Consequently a small aggregate residual RMS +can coexist with an unstable worst-direction coefficient. Deep state +dependence can amplify the same mode across layers or increase the local +covariance until an initially stable negative `k` crosses the lower boundary. This calculation explains the frozen MT-1 failure boundary without changing its result. The training-only audit finds the first active nonfinite tensors @@ -554,6 +563,14 @@ traffic setting. A scalable version needs an operator-level dissipativity or gain-control mechanism; the pretraining residual-RMS threshold is insufficient by itself. +The subsequent frozen S0 training-prefix screen tests a fixed one-sided +predictor margin and finds no eligible value in `{0.001,0.003,0.01,0.03}`. +Every candidate has nonpositive measured neutral residual-soma slope, yet the +smaller margins overflow BatchNorm state and the larger margins show enormous +transient loss and parameter growth. This is consistent with the two-sided +condition above: sign control is necessary but does not bound the evolving +gain `d c` or nonnormal deep-network transients. + ### Intermittent feedback tracking can hide behind a final cosine An idealized mirror event every `k` task updates uses |
