From 2484a7ef7dbf0e0996424a5797aea9002a7c3a52 Mon Sep 17 00:00:00 2001 From: Yuren Hao Date: Sun, 12 Jul 2026 08:33:55 -0500 Subject: Hardware outreach v2: gate lifted, clockless-MVP story, Dillavou wave-1, brief rewritten to cascade-era Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn --- docs/hardware/COLLABORATOR_BRIEF.md | 87 +++++++++++++++++++++---------------- 1 file changed, 49 insertions(+), 38 deletions(-) (limited to 'docs/hardware/COLLABORATOR_BRIEF.md') diff --git a/docs/hardware/COLLABORATOR_BRIEF.md b/docs/hardware/COLLABORATOR_BRIEF.md index 3728657..0ce68ea 100644 --- a/docs/hardware/COLLABORATOR_BRIEF.md +++ b/docs/hardware/COLLABORATOR_BRIEF.md @@ -1,46 +1,57 @@ -# Backprop-free analog training of a transformer — collaboration brief -**One-page ask for hardware-side collaborators · 2026-06-21 · Yuren Hao (UIUC)** +# Backprop-free training of transformers — results and a staged hardware program +**One-page brief for hardware-side collaborators · rev. 2026-07-12 · Yuren Hao (UIUC)** ## The idea in three sentences -We train a **transformer block as a physical equilibrium (fixed-point) system** using **Equilibrium Propagation -(EP)** — no backpropagation. The forward pass is a damped relaxation `z ← z + ε·F(z)` that **settles** to a fixed -point (on analog hardware, the settling *is* the physics — nearly free); the weight update is **local**, computed -from the contrast between a free settle and a slightly-nudged settle. This is exactly the computation an analog -in-memory / memristive array is good at — and unlike every shipping analog-AI chip (all inference-only), it needs -**in-situ weight update**, which is the open opportunity. +We train **standard multi-layer transformers** with **Equilibrium Propagation** on a layered energy: +training consists of two relaxation phases and a **local** contrast update per weight — no +backpropagation anywhere — and inference is an ordinary forward pass. On GPU this now works at +language-model scale with essentially no quality gap to backprop. The hardware program starts with +the cheapest object that can validate the physical learning rule — a **clockless twin-network +analog tile (~$300)** — and climbs rung by rung to an in-memory-compute transformer block. -## Why now / why it's real (not speculative) -- **Algorithm side (ours, in simulation):** EP's gradient matches true backprop (cosine ≈ 0.99–1.0 per component); - the equilibrium transformer trains stably and **matches/beats a same-parameter BP transformer** on language modeling. - Currently scaling the recipe; a fix for the one known instability (a residual-defense term) is under validation. -- **Hardware precedent exists:** local contrastive/EP learning has been physically demonstrated (self-learning analog - resistor networks, ~1 µs settling, on-chip weight update from a local free-vs-clamped difference; EP on a D-Wave - Ising machine). **But nobody has built an EP-trained *transformer* in analog hardware — that is the first-mover demo.** -- **Endurance clears the bar:** HfOx-class RRAM survives ~10^10 write cycles; a training run needs ≤10^8 device writes - (fewer with digital-accumulate-then-threshold-program). Endurance is not the blocker — update linearity/symmetry is - the real device challenge. +## GPU-scale results (2026-07, measured) +- **A 12-layer, 42.7M-parameter transformer LM trained for a full epoch (59k steps, 361M tokens) + with no backpropagation in the training loop; it generates coherent text.** To our knowledge the + first transformer language model trained fully this way. +- **Gap to a tuned, same-architecture backprop control: 0.05 nats** (at 4k steps: statistically + indistinguishable, 3 seeds/arm). Prior backprop-free attempts at scale all report qualitative gaps. +- EP step = **3.2× backprop FLOPs** (measured); mixed-precision training validated; the two known + EP-specific instabilities are mechanistically diagnosed and closed (an estimator-SNR floor with a + β-schedule law; a relaxation-contractivity crossing eliminated by norm placement). +- **Every trained operation chosen analog-implementable:** crossbar MVM, divisive normalization, + fixed I/Q rotations (position code), translinear gated MLP, subthreshold-exponential softmax, + two-phase relaxation for the learning rule. -## What a hardware demo needs (three layers) — and the UIUC ECE fit -| Layer | What it does | Closest collaborator | -|---|---|---| -| **Trainable device** | in-situ-updatable analog weights (RRAM/FeFET/ECRAM) — *the part you cannot buy* | **Wenjuan Zhu** (UIUC ECE, memristor/RRAM/FeFET/2D devices) | -| **In-memory MVM circuit** | analog matrix-vector multiply + on-chip weight write-back | **Naresh Shanbhag** (UIUC ECE) — his JSSC-2018 DIMA chip *already* does analog MVM **+ on-chip SGD weight write-back** in 65nm; nearest existing substrate | -| **Mixed-signal glue / control loop** | ADC/DAC to read settled states + apply the nudge; switched-cap integrators = relaxation primitives | **Pavan Hanumolu** (UIUC ECE, data converters / PLL / switched-cap) | -| **EP control + sim** | the settle→nudge→settle→local-Δθ loop, noise/endurance de-risk in simulation | **us** (FPGA + the trained model + analog-noise sim already built) | +## Measured fault tolerances (fault injection at the trained model) +| fault | free | marginal | dead | +|---|---|---|---| +| weight precision | **8-bit** (ΔCE +0.004) | 6-bit (+0.05) | 4-bit | +| forward state noise | **1%** | — | — | +| error-channel (nudge) noise | **10% relative** | 30% | — | +| divider mismatch / gate gain / phase error | 3% / 10% / 0.03 rad | 10% / — / 0.1 rad | — | -**Escalation / device frontier:** **H.-S. Philip Wong (黄汉森, Stanford EE / TSMC Chief Scientist)** — NeuRRAM (Nature -2022) is the most EP-relevant analog-MVM substrate (inference-only today); the RRAM-device heavyweight + a TSMC-foundry -path, reachable via a Stanford student contact. +Under every non-fatal fault the learning signal tracks the *faulted* network (gradient cosine ≈ 0.97 +invariant): **the rule co-adapts to the device.** The only hard spec is ~7-bit effective weights. -## The concrete ask (staged, modular — stitch existing capabilities, no startup-scale custom fab) -- **Phase 1:** put ONE equilibrium-transformer block on an existing in-situ-trainable substrate (Shanbhag's DIMA-class - chip + Hanumolu converter/integrator glue; Zhu devices) + our FPGA EP-control loop → prove end-to-end analog EP training. -- **Phase 2:** scale weights (foundry RRAM MPW — e.g. SkyWater S130 + Weebit ReRAM IP — or a fixed-weight inference array - for the forward path with the trainable layer in-situ). -- **What we bring:** the validated algorithm, the trained model + scaling data, the EP control logic, and a simulator - that already models analog non-idealities (device noise / quantization / asymmetric update) to de-risk before tape-out. +## The hardware ladder (each rung publishable alone) +1. **One-edge metrology tile ($70–130):** twin MOSFET edge, shared weight capacitor, exact + (difference-of-squares) and sign-only local update channels, OTA current nudge — no processor, + converter, clock, or sampled memory in the learning loop. +2. **8-edge twin network ($170–300):** nonlinear regression; EP current-nudge vs Coupled-Learning + voltage-clamp on one board; exact-vs-sign update comparison; measured bias-vs-nudge-magnitude + curve (the same β-SNR law we measured in simulation). +3. **32-edge network ($450–900):** replication-class nonlinear tasks, robustness study. +4. **Reciprocal attention microcell (+$100–250):** two tokens, one head, energy-based attention. +5. **CIM transformer block (partner phase):** analog MVM + in-situ two-phase EP weight update — + the piece no shipping analog-AI chip has (all are inference-only or on-chip-backprop). +6. **North star: a few-M-parameter TinyStories LM trained on analog hardware.** -**Bottom line:** the science is done in sim and the hardware pieces all exist in-house at UIUC ECE — this is a -stitching + first-demo opportunity, not a multi-year custom-silicon program. +## What we bring / what we ask +**Bring:** the trained models and recipe, the estimator theory (β-SNR law, stability walls), the +measured tolerance ledger, SPICE-first costed build plan, and parts funding (rungs 1–3 are <$1k). +**Ask (rungs 1–3):** bench access, analog-design mentorship, and/or a student who enjoys discrete +analog — six-week plan, instruments = a scope and a DMM. **Ask (rung 5):** a CIM/mixed-signal +partnership where the substrate expertise is yours and the learning rule is ours. -*(Backing detail + citations: HW_RESEARCH_FINDINGS.md; method: ept_method_intro.pdf)* +*(Detail: CLOCKLESS_ANALOG_MVP_PLAN.md — full BOM, schedule, acceptance criteria, claim limits; +COMPONENT_HW_MAP.md — per-operation analog mapping + tolerance status.)* -- cgit v1.2.3