Hardware outreach v2: gate lifted, clockless-MVP story, Dillavou wave-1, brief rewritten to cascade-era

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn

2 files changed, 55 insertions, 39 deletions

diff --git a/docs/hardware/COLLABORATOR_BRIEF.md b/docs/hardware/COLLABORATOR_BRIEF.md
index 3728657..0ce68ea 100644
--- a/docs/hardware/COLLABORATOR_BRIEF.md
+++ b/docs/hardware/COLLABORATOR_BRIEF.md
@@ -1,46 +1,57 @@
-# Backprop-free analog training of a transformer — collaboration brief
-**One-page ask for hardware-side collaborators · 2026-06-21 · Yuren Hao (UIUC)**
+# Backprop-free training of transformers — results and a staged hardware program
+**One-page brief for hardware-side collaborators · rev. 2026-07-12 · Yuren Hao (UIUC)**
 
 ## The idea in three sentences
-We train a **transformer block as a physical equilibrium (fixed-point) system** using **Equilibrium Propagation
-(EP)** — no backpropagation. The forward pass is a damped relaxation `z ← z + ε·F(z)` that **settles** to a fixed
-point (on analog hardware, the settling *is* the physics — nearly free); the weight update is **local**, computed
-from the contrast between a free settle and a slightly-nudged settle. This is exactly the computation an analog
-in-memory / memristive array is good at — and unlike every shipping analog-AI chip (all inference-only), it needs
-**in-situ weight update**, which is the open opportunity.
+We train **standard multi-layer transformers** with **Equilibrium Propagation** on a layered energy:
+training consists of two relaxation phases and a **local** contrast update per weight — no
+backpropagation anywhere — and inference is an ordinary forward pass. On GPU this now works at
+language-model scale with essentially no quality gap to backprop. The hardware program starts with
+the cheapest object that can validate the physical learning rule — a **clockless twin-network
+analog tile (~$300)** — and climbs rung by rung to an in-memory-compute transformer block.
 
-## Why now / why it's real (not speculative)
-- **Algorithm side (ours, in simulation):** EP's gradient matches true backprop (cosine ≈ 0.99–1.0 per component);
-  the equilibrium transformer trains stably and **matches/beats a same-parameter BP transformer** on language modeling.
-  Currently scaling the recipe; a fix for the one known instability (a residual-defense term) is under validation.
-- **Hardware precedent exists:** local contrastive/EP learning has been physically demonstrated (self-learning analog
-  resistor networks, ~1 µs settling, on-chip weight update from a local free-vs-clamped difference; EP on a D-Wave
-  Ising machine). **But nobody has built an EP-trained *transformer* in analog hardware — that is the first-mover demo.**
-- **Endurance clears the bar:** HfOx-class RRAM survives ~10^10 write cycles; a training run needs ≤10^8 device writes
-  (fewer with digital-accumulate-then-threshold-program). Endurance is not the blocker — update linearity/symmetry is
-  the real device challenge.
+## GPU-scale results (2026-07, measured)
+- **A 12-layer, 42.7M-parameter transformer LM trained for a full epoch (59k steps, 361M tokens)
+  with no backpropagation in the training loop; it generates coherent text.** To our knowledge the
+  first transformer language model trained fully this way.
+- **Gap to a tuned, same-architecture backprop control: 0.05 nats** (at 4k steps: statistically
+  indistinguishable, 3 seeds/arm). Prior backprop-free attempts at scale all report qualitative gaps.
+- EP step = **3.2× backprop FLOPs** (measured); mixed-precision training validated; the two known
+  EP-specific instabilities are mechanistically diagnosed and closed (an estimator-SNR floor with a
+  β-schedule law; a relaxation-contractivity crossing eliminated by norm placement).
+- **Every trained operation chosen analog-implementable:** crossbar MVM, divisive normalization,
+  fixed I/Q rotations (position code), translinear gated MLP, subthreshold-exponential softmax,
+  two-phase relaxation for the learning rule.
 
-## What a hardware demo needs (three layers) — and the UIUC ECE fit
-| Layer | What it does | Closest collaborator |
-|---|---|---|
-| **Trainable device** | in-situ-updatable analog weights (RRAM/FeFET/ECRAM) — *the part you cannot buy* | **Wenjuan Zhu** (UIUC ECE, memristor/RRAM/FeFET/2D devices) |
-| **In-memory MVM circuit** | analog matrix-vector multiply + on-chip weight write-back | **Naresh Shanbhag** (UIUC ECE) — his JSSC-2018 DIMA chip *already* does analog MVM **+ on-chip SGD weight write-back** in 65nm; nearest existing substrate |
-| **Mixed-signal glue / control loop** | ADC/DAC to read settled states + apply the nudge; switched-cap integrators = relaxation primitives | **Pavan Hanumolu** (UIUC ECE, data converters / PLL / switched-cap) |
-| **EP control + sim** | the settle→nudge→settle→local-Δθ loop, noise/endurance de-risk in simulation | **us** (FPGA + the trained model + analog-noise sim already built) |
+## Measured fault tolerances (fault injection at the trained model)
+| fault | free | marginal | dead |
+|---|---|---|---|
+| weight precision | **8-bit** (ΔCE +0.004) | 6-bit (+0.05) | 4-bit |
+| forward state noise | **1%** | — | — |
+| error-channel (nudge) noise | **10% relative** | 30% | — |
+| divider mismatch / gate gain / phase error | 3% / 10% / 0.03 rad | 10% / — / 0.1 rad | — |
 
-**Escalation / device frontier:** **H.-S. Philip Wong (黄汉森, Stanford EE / TSMC Chief Scientist)** — NeuRRAM (Nature
-2022) is the most EP-relevant analog-MVM substrate (inference-only today); the RRAM-device heavyweight + a TSMC-foundry
-path, reachable via a Stanford student contact.
+Under every non-fatal fault the learning signal tracks the *faulted* network (gradient cosine ≈ 0.97
+invariant): **the rule co-adapts to the device.** The only hard spec is ~7-bit effective weights.
 
-## The concrete ask (staged, modular — stitch existing capabilities, no startup-scale custom fab)
-- **Phase 1:** put ONE equilibrium-transformer block on an existing in-situ-trainable substrate (Shanbhag's DIMA-class
-  chip + Hanumolu converter/integrator glue; Zhu devices) + our FPGA EP-control loop → prove end-to-end analog EP training.
-- **Phase 2:** scale weights (foundry RRAM MPW — e.g. SkyWater S130 + Weebit ReRAM IP — or a fixed-weight inference array
-  for the forward path with the trainable layer in-situ).
-- **What we bring:** the validated algorithm, the trained model + scaling data, the EP control logic, and a simulator
-  that already models analog non-idealities (device noise / quantization / asymmetric update) to de-risk before tape-out.
+## The hardware ladder (each rung publishable alone)
+1. **One-edge metrology tile ($70–130):** twin MOSFET edge, shared weight capacitor, exact
+   (difference-of-squares) and sign-only local update channels, OTA current nudge — no processor,
+   converter, clock, or sampled memory in the learning loop.
+2. **8-edge twin network ($170–300):** nonlinear regression; EP current-nudge vs Coupled-Learning
+   voltage-clamp on one board; exact-vs-sign update comparison; measured bias-vs-nudge-magnitude
+   curve (the same β-SNR law we measured in simulation).
+3. **32-edge network ($450–900):** replication-class nonlinear tasks, robustness study.
+4. **Reciprocal attention microcell (+$100–250):** two tokens, one head, energy-based attention.
+5. **CIM transformer block (partner phase):** analog MVM + in-situ two-phase EP weight update —
+   the piece no shipping analog-AI chip has (all are inference-only or on-chip-backprop).
+6. **North star: a few-M-parameter TinyStories LM trained on analog hardware.**
 
-**Bottom line:** the science is done in sim and the hardware pieces all exist in-house at UIUC ECE — this is a
-stitching + first-demo opportunity, not a multi-year custom-silicon program.
+## What we bring / what we ask
+**Bring:** the trained models and recipe, the estimator theory (β-SNR law, stability walls), the
+measured tolerance ledger, SPICE-first costed build plan, and parts funding (rungs 1–3 are <$1k).
+**Ask (rungs 1–3):** bench access, analog-design mentorship, and/or a student who enjoys discrete
+analog — six-week plan, instruments = a scope and a DMM. **Ask (rung 5):** a CIM/mixed-signal
+partnership where the substrate expertise is yours and the learning rule is ours.
 
-*(Backing detail + citations: HW_RESEARCH_FINDINGS.md; method: ept_method_intro.pdf)*
+*(Detail: CLOCKLESS_ANALOG_MVP_PLAN.md — full BOM, schedule, acceptance criteria, claim limits;
+COMPONENT_HW_MAP.md — per-operation analog mapping + tolerance status.)*
diff --git a/docs/hardware/COMPONENT_HW_MAP.md b/docs/hardware/COMPONENT_HW_MAP.md
index ff47123..0cae1ea 100644
--- a/docs/hardware/COMPONENT_HW_MAP.md
+++ b/docs/hardware/COMPONENT_HW_MAP.md
@@ -48,7 +48,12 @@ Companion docs: `HW_RESEARCH_FINDINGS.md` (softmax dossier, OLMo2 analog audit,
 
 ## D. Demo scoping (reuse-doctrine currency)
 
-- **Demo-0 (spec target of E-tier):** single cascade block, SRAM-CIM eval module + FPGA harness +
+- **⚡ 2026-07-12: Demo-0 SUPERSEDED as the first rung by CLOCKLESS_ANALOG_MVP_PLAN.md**
+  (user-authored): a $170–300 clockless twin-network tile (Dillavou-lineage; EP current-nudge vs CL
+  voltage-clamp; exact vs sign local update; no processor/converter/clock in the learning loop)
+  validates the physical learning primitive first. The CIM block below becomes the PARTNER rung
+  (rung 5 of the ladder in COLLABORATOR_BRIEF.md).
+- **Demo-0 (now rung 5, partner phase):** single cascade block, SRAM-CIM eval module + FPGA harness +
   COTS mixer/DAC/ADC periphery; goal = demonstrate the two-phase θ-read on one block with measured
   tolerances. Board-BOM class: $5–20k.
 - **Demo-1:** time-multiplex ONE physical trainable block through L logical positions (cascade's