1 files changed, 63 insertions, 0 deletions
diff --git a/docs/hardware/HW_RESEARCH_FINDINGS.md b/docs/hardware/HW_RESEARCH_FINDINGS.md
index c63106a..1e040c9 100644
--- a/docs/hardware/HW_RESEARCH_FINDINGS.md
+++ b/docs/hardware/HW_RESEARCH_FINDINGS.md
@@ -132,3 +132,66 @@ parameter at LM scale.
 PAR process — position the demo as "PAR learning + our spectral/residual stability control on
 non-reciprocal analog hardware": complementary to (and citing) their paper, covering exactly the half
 they don't (forward stability), on their home turf.
+
+---
+
+## 2026-07-10 — ANALOG SOFTMAX / ATTENTION DOSSIER (deep-research, 12 claims 3-0 verified; synthesis truncated by API limit)
+
+**Verdict: the whole cascade-analog hypothesis chain is CONFIRMED, and the training gap is REAL.**
+Every non-weight op of attention has a demonstrated analog primitive; nobody has trained attention in analog.
+
+### (a) softmax = subthreshold-exp + common-node KCL normalizer — CONFIRMED, and reciprocal
+- **Elfadel & Wyatt, NIPS 1993** (`proceedings.neurips.cc/paper/1993/hash/352407221afb776e3143e8a1a0577885`):
+  explicit subthreshold-CMOS softmax, i_m = I_c·exp(κv_m/V_0) / Σ exp(κv_p/V_0). exp from subthreshold
+  MOS drain current; normalization from KCL at a shared node fed by control current I_c. **Exactly our
+  hypothesis.** Temperature/process enter via V_0=kT/q and κ (calibration target).
+- **★ Reciprocity (KEY for EP):** their Theorem 1 — the *diode-connected* variant ("entropic resistor")
+  is **reciprocal, passive, locally passive**, with log-sum-exp co-content Φ(v)=(1/κ)I_c V_0 ln Σexp(κv_m/V_0).
+  The naive gate-in/drain-out softmax is NOT reciprocal and can't compose into energy-based networks.
+  **⟹ there exists a softmax circuit that composes into reciprocal EP optimization networks.** This is
+  the physical realization of "softmax Jacobian diag(p)−ppᵀ is symmetric ⟹ VJP needs no adjoint" — our
+  PAR-wall escape for attention is not just algebraic, it has a named 1993 circuit.
+- **Softmax Jacobian symmetry** confirmed from the same paper (DF = diag(F)−FFᵀ, symmetric, singular,
+  eigenvalues < 1, softmax = ∇ of a convex potential).
+- **Gilbert 1984** 16-channel analog array normalizer (IEEE JSSC SC-19(6):956): divisive normalization
+  in silicon since 1984.
+- **Lazzaro WTA 1988** (the reference I'd assumed) is actually a HARD max (argmax + log(max)), NOT
+  softmax — cite Elfadel&Wyatt for soft, Lazzaro only for the common-node inhibition mechanism.
+- **Modern (2023, arXiv:2305.13649):** differential-pair softmax, tail-current-normalized. Measured
+  ±0.9% error (subthreshold NMOS, SPICE, N=4); breadboard ±4.2%. Only N=4 ever built — **large N is
+  open**, our first tolerance question.
+- **★ Tolerance transfer coefficient (plug into E-tier sim):** ±1% device (group-constant Δc) mismatch
+  → **≤±1% softmax output error, LINEAR.** (arXiv:2305.13649.) Pelgrom σ(ΔVT)=A_VT/√(WL) is the
+  underlying law (Kinget JSSC 2005).
+
+### (b) act×act (QKᵀ, P·V) with transient operands = charge-domain gain cells — CONFIRMED (built, sim+device)
+- **Leroux et al., Nature Comp. Sci. 2025** (`nature.com/articles/s43588-025-00854-1`, arXiv:2409.19315):
+  gain-cell (capacitor/OSFET, volatile charge) crossbars **simultaneously store the KV cache and compute
+  QKᵀ and score·V in analog.** K,V as capacitor voltages in 64×64 arrays; Q & scores as PWM; products
+  summed as bitline currents. Precision: 4-bit Q, 3-bit stored K/V, 5-bit out. **This is our hypothesis
+  (b) already built.** BUT they **replaced softmax with HardSigmoid** (charge-to-pulse), citing
+  sigmoid≈softmax attention up to ~7B — i.e. they sidestepped analog softmax, didn't solve it.
+- Memristor route (Bettayeb, Sci.Rep. 2024) hits **write-endurance** limits writing transient operands
+  into RRAM → confirms crossbars are wrong for act×act; **charge/capacitor is the right primitive** (KV
+  is volatile anyway).
+
+### (c) LayerNorm / divisive normalization = Gilbert normalizer / RMS circuits — prior art exists (as above).
+
+### THE GAP — CONFIRMED unclaimed
+Every analog/photonic attention accelerator surveyed **offloads softmax to digital or replaces it**, and
+**all are inference-only** (weights trained offline in software, mapped to conductances):
+Leroux 2025 (HardSigmoid + software adapt), 65nm hybrid chip arXiv:2409.04940 (softmax+PV digital),
+Bettayeb 2024 (exp/log via digital LUT), photonic Lightening-Transformer HPCA2024 & arXiv:2603.12934
+(GEMMs optical, softmax electronic). **No one has TRAINED softmax attention in analog with any learning
+rule.** Pre-transformer analog neuro had no softmax demand; post-transformer analog work is inference-only.
+⟹ "standard-form transformer, softmax intact, trained on analog by a local two-phase rule" is open territory.
+
+### EP-in-crossbars precedent (RQ6): Kendall/Scellier/Bengio 2020 (arXiv:2006.01981) — nonlinear
+resistive networks (memristor + diodes) are EBMs by Kirchhoff, trainable by EP with local rules; reverse
+crossbar drive = Wᵀ. (Kendall → Rain AI.) Our reciprocal "entropic resistor" softmax slots into exactly
+this framework.
+
+**Open for the deferred optimizer-state dossier (wceo0z7fy, hit API limit — rerun later):** analog
+momentum (leaky integrator / Loihi eligibility trace), Adam-as-AGC (translinear squarer + RC + divider),
+IBM Tiki-Taka aux-array momentum, and the Muon-spectral gap. Hypothesis: Adafactor (row/col shared 2nd
+moment = per-line AGC) is the analog-native adaptive optimizer.