summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 16:56:13 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 16:56:13 -0500
commit26c257fa528118671a5914f9f9796b24d451839f (patch)
tree8b25903e21b6e51cf4885f049637d115ab28c059
parent8d28fd9f9e85f5e0715b3be2337b7843fc4c943f (diff)
theory: bound both sides of residual stability
-rw-r--r--THEORY.md27
-rw-r--r--experiments/verify_theory.py5
2 files changed, 27 insertions, 5 deletions
diff --git a/THEORY.md b/THEORY.md
index 8299c75..f659d0f 100644
--- a/THEORY.md
+++ b/THEORY.md
@@ -539,11 +539,20 @@ joint eigenmode let `k=d c-lambda`, where `d` and `c>=0` are eigenvalues of
The determinant is `mu`. Evaluating its characteristic polynomial at one
gives `p(1)=-eta k`; therefore every `k>0` produces a real eigenvalue above
-one. Weight decay stabilizes the mode only if it makes the effective `k`
-nonpositive, with the usual additional discrete-time step-size constraints.
-Consequently a small aggregate residual RMS can coexist with an unstable
-worst-direction coefficient. Deep state dependence can amplify the same mode
-across layers before a slow neutral predictor catches it.
+one. The full second-order Jury conditions give the strict stability window
+
+```text
+|mu| < 1,
+-2 (1 + mu) / eta < k < 0.
+```
+
+The upper boundary excludes positive Hebbian feedback; the lower boundary
+excludes an over-strong negative mode that alternates and grows. Weight decay
+or predictor overestimation must therefore control both sign and magnitude,
+not merely make `k` nonpositive. Consequently a small aggregate residual RMS
+can coexist with an unstable worst-direction coefficient. Deep state
+dependence can amplify the same mode across layers or increase the local
+covariance until an initially stable negative `k` crosses the lower boundary.
This calculation explains the frozen MT-1 failure boundary without changing
its result. The training-only audit finds the first active nonfinite tensors
@@ -554,6 +563,14 @@ traffic setting. A scalable version needs an operator-level dissipativity or
gain-control mechanism; the pretraining residual-RMS threshold is
insufficient by itself.
+The subsequent frozen S0 training-prefix screen tests a fixed one-sided
+predictor margin and finds no eligible value in `{0.001,0.003,0.01,0.03}`.
+Every candidate has nonpositive measured neutral residual-soma slope, yet the
+smaller margins overflow BatchNorm state and the larger margins show enormous
+transient loss and parameter growth. This is consistent with the two-sided
+condition above: sign control is necessary but does not bound the evolving
+gain `d c` or nonnormal deep-network transients.
+
### Intermittent feedback tracking can hide behind a final cosine
An idealized mirror event every `k` task updates uses
diff --git a/experiments/verify_theory.py b/experiments/verify_theory.py
index ad0e9b9..d4325e9 100644
--- a/experiments/verify_theory.py
+++ b/experiments/verify_theory.py
@@ -220,18 +220,23 @@ def check_residual_coupling_instability():
positive_k = 0.02
negative_k = -0.20
+ too_negative_k = -40.0
positive_radius = radius(positive_k)
negative_radius = radius(negative_k)
+ too_negative_radius = radius(too_negative_k)
polynomial_at_one = -eta * positive_k
print("\nMULTIPLICATIVE RESIDUAL COUPLING")
print(f"vectorization_error={identity_error:.3e} "
f"rho(k={positive_k:+.3f})={positive_radius:.9f} "
f"rho(k={negative_k:+.3f})={negative_radius:.9f} "
+ f"rho(k={too_negative_k:+.1f})={too_negative_radius:.9f} "
f"p_positive(1)={polynomial_at_one:+.3e}")
assert identity_error < 2e-15
assert polynomial_at_one < 0.0
assert positive_radius > 1.0
assert negative_radius < 1.0
+ assert too_negative_k < -2.0 * (1.0 + momentum) / eta
+ assert too_negative_radius > 1.0
def check_intermittent_feedback_tracking():