diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-15 13:40:14 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-15 13:40:14 -0500 |
| commit | 82fcd64617f02156762dea257dbff3fc0316613a (patch) | |
| tree | 914449bfa4981d758a7f9716214b745746b4de1d /docs | |
| parent | a6d88a18a4bfd46ab0dd487730b72dcbc3b8b5b8 (diff) | |
External notes: sliding-mode clamping (Stage-0 injection rule), saturated-unit EP-BPTT divergence (smooth-gate methods rule), momentum-bias unification with RESULT 22
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 19 |
1 files changed, 19 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index b958574..555f803 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -568,6 +568,25 @@ direction), not training-under-fault; wave-2 = co-training with faults injected +### EXTERNAL NOTES (2026-07-15, user-relayed from a sibling Hopfield-EP library project): +1. Hard-sigmoid Hopfield needs state CLAMPING: units with drive>1 equilibrate exactly ON the + rho breakpoint (sliding-mode equilibrium); unclamped discrete Euler chatters around it with + O(1) amplitude forever. -> Taxonomy exhibit #3 for the dynamics paper (solver-artifact family: + Hopf / loop-gain / sliding-mode chatter). -> OPERATIONAL RULE for Stage-0 fault injection: + when injecting rail/saturation, CLAMP states — otherwise you measure integrator chatter, + not device physics. +2. Saturated units make finite-beta EP legitimately diverge from BPTT (measured 80-130 deg): + dead units are invisible to BP, the nudge can revive them. NOT a bug — a hard-rho property. + -> METHODS RULE: gradient-equivalence gates require smooth activations (our gates comply). + -> Discussion candidate: finite-beta as exploration/repair (dead-head revival probe, low prio). + -> HW note: at device rails, EP's systematic deviation points OFF the rail — likely a + robustness bonus for analog training. +3. Classic EP + momentum collapses (MNIST 85% -> 48%; plain SGD required): momentum integrates + the estimator's non-zero-mean bias. UNIFIES with RESULT 22's dose-response: bias/signal large + -> fatal (their setting); small (governed beta + centered, mom 0.95) -> harmless; pushed to + 0.99+ at low-SNR tail -> harmful again (+0.019/+0.059/+0.076). Momentum tolerance is NOT an + EP property — it is purchased by bias control. + ### RESULT 25 (2026-07-15): ERROR-SOURCE DECOMPOSITION — the bias lives ENTIRELY in between-block transmission; within-block reads are exactly lossless. Probe: probe_blockcos.py (GPU0, s45000 C512 ckpt, 4 batches, fp32). Exact factorization {anchor: free/nudged} x {cotangent: exact-c/EP-transmitted-d}; all four corners share one code |