HW: optimizer BoM audit per cascade block — Adam analog-compatible via factored-AGC (+2x memory, zero ADC), Muon priced out of analog (matrix-matrix NS + full-gradient ADC tax), Lion native baseline

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn

1 files changed, 51 insertions, 0 deletions

diff --git a/docs/hardware/HW_RESEARCH_FINDINGS.md b/docs/hardware/HW_RESEARCH_FINDINGS.md
index 1e040c9..d380b39 100644
--- a/docs/hardware/HW_RESEARCH_FINDINGS.md
+++ b/docs/hardware/HW_RESEARCH_FINDINGS.md
@@ -195,3 +195,54 @@ this framework.
 momentum (leaky integrator / Loihi eligibility trace), Adam-as-AGC (translinear squarer + RC + divider),
 IBM Tiki-Taka aux-array momentum, and the Muon-spectral gap. Hypothesis: Adafactor (row/col shared 2nd
 moment = per-line AGC) is the analog-native adaptive optimizer.
+
+---
+
+## 2026-07-11 — OPTIMIZER BoM AUDIT: what each cascade block needs in analog, Adam vs Muon (vs Lion baseline)
+
+Per-block inventory for one standard transformer block, cascade-EP training. Weight count per block
+= 12C² (QKV 3C², out C², FFN 8C²). Numbers at C=512 (D1 scale, 3.1M weights/block) and C=1536
+(sub-B scale, 28M weights/block). [dossier] = literature-anchored; [est] = engineering estimate.
+
+### Common substrate (optimizer-independent)
+| item | count per block | primitive |
+|---|---|---|
+| weight crossbars | 12C² cells | memristor/PCM, reciprocal → transpose free [dossier: Kendall 2020] |
+| KV charge arrays | 2·T·C gain cells (volatile) | Leroux 2025 gain-cell tiles [dossier] |
+| softmax normalizers | H per-row common-node circuits, T branches each (time-muxed) | Elfadel-Wyatt entropic resistor [dossier] |
+| LayerNorm | 2 divisive-norm/RMS circuits, C channels | Gilbert 1984 normalizer family [dossier] |
+| bidirectional periphery | 2× crossbar drivers/sense (reverse drive for Jᵀ) | [dossier: analog-BP/EP practice] |
+| outer-product write | pulse-coincidence programming per crossbar | standard analog-training periphery |
+
+### ADAM per block
+| ingredient | analog form | added components | verdict |
+|---|---|---|---|
+| momentum m | per-synapse leaky integrator (cap+leak) OR Tiki-Taka fast aux array | **+12C² cells (+100% analog memory)** — 3.1M @C512, 28M @C1536 | feasible; aux-array form is proven practice [Tiki-Taka; pending re-research] |
+| second moment v | per-synapse squarer = economic suicide; **factored (Adafactor-style) row/col statistics** | ~16C AGC channels (translinear squarer + RC + divider, ~10–30 transistors + 1 cap each) ≈ 8k channels @C512 | feasible [est]; "Adafactor is the analog-native Adam" |
+| m/√v division | per-LINE programmable driver gain (factored) | C-channel gain DACs per matrix | feasible [est] |
+| bias correction / schedules | digital global scalars | negligible | — |
+| **total Adam tax** | | **~2× analog memory + ~16C AGC + per-line gain** ; ZERO gradient digitization | **the analog-compatible adaptive optimizer** |
+| precision risk | AGC mismatch = per-line lr noise (linear transfer [dossier: ±1%→±1%]) | needs E-tier arm | |
+
+### MUON per block
+| ingredient | analog form | added components | verdict |
+|---|---|---|---|
+| momentum | same as Adam | +12C² cells | feasible |
+| Newton-Schulz (X↦aX+b(XXᵀ)X+…, 5 iters) | matrix-MATRIX on a transient matrix = ~10–15·C vector-passes per matrix per step; needs a second C×C act×act fabric per matrix or steals the KV arrays | ~1000× the training step's own MVM duty [est, matches GPU-side analysis] | **analog: economically dead** |
+| mixed-signal Muon | digitize FULL gradient/momentum matrix every step: 12C² conversions @ 8–16 bit per block per step (3.1M @C512, 37M/step for L12) + digital NS matmul sidecar + full-precision DAC write-back | ADC energy dominates; this is the exact cost Lion's 1-bit updates avoid | **negates the analog-training story; Muon stays a GPU/digital-era tool** |
+
+### LION baseline (for contrast)
+momentum array (+12C²) + 1-bit sign comparators per line + fixed-amplitude pulse writes.
+**Minimal periphery, zero ADC traffic, native to pulse programming.** Known quality tax vs AdamW
+~15–20% PPL-relative [OmniOpt @1B]; MARS-Lion recovers part.
+
+### Cost ladder (per block, relative)
+```
+SGD/Lion:   1.0× memory-cells, ~0 ADC/step, trivial periphery      ← analog-native
+Adam(fact): 2.0× memory-cells, ~0 ADC/step, +16C AGC channels      ← analog-compatible
+Muon:       2.0× memory-cells, +12C² ADC conversions/step + digital NS sidecar ← mixed-signal heavy
+```
+Recommendation for the chip narrative: **Lion (or SGDM) as the on-chip rule; factored-Adam as the
+premium tier; Muon explicitly priced out** (its value is in the GPU/simulation era, where it is
+nearly free). Pending verification: Tiki-Taka transfer constants, AGC channel specs, Loihi-trace
+leakage numbers — the deferred optimizer-state deep-research (rerun when API budget recovers).