diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-09 23:10:39 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-09 23:10:39 -0500 |
| commit | 7ea3e8f1a688b8da9aed5623c8d12250594b2336 (patch) | |
| tree | 74eb4e1b05fd7936f9ac2fc43d03a7c829c9f1fa | |
| parent | 813235213786e4e9e6a6fd81f5a2d7db902b650f (diff) | |
HW BoM correction: Lion/SGD ruled out for EP training (empirical, abl_lion2 blew @600 + prior tests) — factored-Adam is the MANDATORY analog baseline, not premium tier
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
| -rw-r--r-- | docs/hardware/HW_RESEARCH_FINDINGS.md | 13 |
1 files changed, 9 insertions, 4 deletions
diff --git a/docs/hardware/HW_RESEARCH_FINDINGS.md b/docs/hardware/HW_RESEARCH_FINDINGS.md index d380b39..e6b77b9 100644 --- a/docs/hardware/HW_RESEARCH_FINDINGS.md +++ b/docs/hardware/HW_RESEARCH_FINDINGS.md @@ -242,7 +242,12 @@ SGD/Lion: 1.0× memory-cells, ~0 ADC/step, trivial periphery ← analog-n Adam(fact): 2.0× memory-cells, ~0 ADC/step, +16C AGC channels ← analog-compatible Muon: 2.0× memory-cells, +12C² ADC conversions/step + digital NS sidecar ← mixed-signal heavy ``` -Recommendation for the chip narrative: **Lion (or SGDM) as the on-chip rule; factored-Adam as the -premium tier; Muon explicitly priced out** (its value is in the GPU/simulation era, where it is -nearly free). Pending verification: Tiki-Taka transfer constants, AGC channel specs, Loihi-trace -leakage numbers — the deferred optimizer-state deep-research (rerun when API budget recovers). +**CORRECTION (2026-07-11, project empirical record): Lion/SGD are RULED OUT for EP training** — +tested and failed (looped line: abl_lion2 diverged at step 600, val 20.1 / res 0.34; earlier tests +concur). Per-parameter adaptivity (Adam-class) is empirically LOAD-BEARING for EP gradients here. +⟹ Revised chip narrative: **factored-Adam (AGC/Tiki-Taka route) is not the premium tier — it is +the MANDATORY baseline**. The +2× analog memory and ~16C AGC channels per block are the price of +admission for analog EP training, and the OmniOpt "Lion is cheap"路线 does not apply to us. +Muon stays priced out of analog (GPU/simulation-era tool). This RAISES the priority of the deferred +optimizer-state research (Tiki-Taka transfer constants, AGC channel specs, trace leakage — rerun +when API budget recovers): the analog-Adam engineering is now on the critical path of the chip story. |