summaryrefslogtreecommitdiff
path: root/docs/hardware/COMPONENT_HW_MAP.md
diff options
context:
space:
mode:
authorYuren Hao <yurenh2@illinois.edu>2026-07-12 05:44:18 -0500
committerYuren Hao <yurenh2@illinois.edu>2026-07-12 05:44:18 -0500
commit411870d64f00181ea41a8059f4b53f5b232447c0 (patch)
tree391b22448b2c2398e7857e3c2af39abc9cd037c2 /docs/hardware/COMPONENT_HW_MAP.md
parent155478b7ffdd9015fde13884b1ebff9b92148a20 (diff)
E-tier wave-1 ledger (RESULT 14 + HW map rows) + cost model v2 recalibrated on measured H200 datapoint
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs/hardware/COMPONENT_HW_MAP.md')
-rw-r--r--docs/hardware/COMPONENT_HW_MAP.md10
1 files changed, 5 insertions, 5 deletions
diff --git a/docs/hardware/COMPONENT_HW_MAP.md b/docs/hardware/COMPONENT_HW_MAP.md
index 95c5689..ff47123 100644
--- a/docs/hardware/COMPONENT_HW_MAP.md
+++ b/docs/hardware/COMPONENT_HW_MAP.md
@@ -11,11 +11,11 @@ Companion docs: `HW_RESEARCH_FINDINGS.md` (softmax dossier, OLMo2 analog audit,
| # | Component | Computes | Analog primitive | Reuse source (no tapeout) | Feedback path (nudged phase) | Tolerance status |
|---|---|---|---|---|---|---|
| 1 | Token embedding | table lookup | **digital** (SRAM lookup + DAC line drivers) | any MCU/FPGA + DAC array | none (input boundary) | n/a |
-| 2 | RoPE | fixed per-position 2×2 rotations on q,k | **I/Q quadrature mixer**: cos/sin DDS + multiplying DACs | century-old RF practice; COTS mixer/DDS chips | transpose = rotation by −θ — **same mixer, negated sin** | phase error ≈ position blur — expect very tolerant; **E-tier item (queued)** |
-| 3 | qkv / attn-proj / SwiGLU w1,w3,w2 / head | matrix–vector products | **crossbar arrays** (the bulk of all compute) | Demo-0/1: **SRAM-CIM (Shanbhag DIMA class)**; alternatives: Mythic flash CIM, IBM HERMES PCM eval | J^T reads = **bidirectional crossbar access** (the known PAR price; standard in every analog-EP proposal) | wq8 safe / wq6 marginal (looped-era static-tolerance); **re-run on cascade in E-tier** |
-| 4 | QK-norm (full-width RMS) | q,k ← q/rms(q)·g | **divisive normalization**: square-law devices + KCL current sum + divider/AGC | same primitive class as softmax normalization periphery | Jacobian symmetric — self-transpose, no extra hardware | divider mismatch/offset — **E-tier item (queued)** |
-| 5 | Softmax (causal) | exp + row-normalize | **Elfadel–Wyatt "entropic resistor"**: subthreshold-exp devices + KCL normalization (log-sum-exp co-content) | classic analog-VLSI circuit family; small arrays demonstrated | softmax Jacobian diag(p)−pp^T is symmetric; the QKV coupling around it is the non-reciprocal part | ±1% device mismatch → ±1% softmax error (linear); QK-norm bounds its input range. Dynamic-noise cliff (fnoise ≥1e-3, looped-era) **re-test on cascade = E-tier** |
-| 6 | SwiGLU gate | silu(w1x) ⊙ (w3x) | sigmoid = differential pair (native); ⊙ = **Gilbert/translinear multiplier** per unit | textbook standard cells | transpose of ⊙ = multiply by the partner signal — cheap | per-multiply mismatch/noise, 1408 wide — **E-tier item (queued)** |
+| 2 | RoPE | fixed per-position 2×2 rotations on q,k | **I/Q quadrature mixer**: cos/sin DDS + multiplying DACs | century-old RF practice; COTS mixer/DDS chips | transpose = rotation by −θ — **same mixer, negated sin** | **E-tier w1: 0.03 rad FREE; 0.1 rad ΔCE +0.012, cos_clean 0.92** — ~2° I/Q phase accuracy suffices (easy) |
+| 3 | qkv / attn-proj / SwiGLU w1,w3,w2 / head | matrix–vector products | **crossbar arrays** (the bulk of all compute) | Demo-0/1: **SRAM-CIM (Shanbhag DIMA class)**; alternatives: Mythic flash CIM, IBM HERMES PCM eval | J^T reads = **bidirectional crossbar access** (the known PAR price; standard in every analog-EP proposal) | **E-tier w1 (cascade): wq8 FREE (ΔCE +0.004, cos_clean 0.96); wq6 MARGINAL (+0.051, 0.81); wq4 DEAD (+2.23)** — ≥7b effective weight precision is the binding spec; wave-2 lever = quant-aware co-training |
+| 4 | QK-norm (full-width RMS) | q,k ← q/rms(q)·g | **divisive normalization**: square-law devices + KCL current sum + divider/AGC | same primitive class as softmax normalization periphery | Jacobian symmetric — self-transpose, no extra hardware | **E-tier w1: 3% mismatch FREE (ΔCE +0.003, cos_clean 0.95); 10% MARGINAL (+0.042, 0.78)** — 3% device matching is routine |
+| 5 | Softmax (causal) | exp + row-normalize | **Elfadel–Wyatt "entropic resistor"**: subthreshold-exp devices + KCL normalization (log-sum-exp co-content) | classic analog-VLSI circuit family; small arrays demonstrated | softmax Jacobian diag(p)−pp^T is symmetric; the QKV coupling around it is the non-reciprocal part | ±1% device mismatch → ±1% softmax error (linear); QK-norm bounds its input range. **E-tier w1: forward additive state noise σ=1e-2 FREE (ΔCE +0.001, cos 0.97) — the looped-era 1e-3 cliff does NOT transfer to cascade** |
+| 6 | SwiGLU gate | silu(w1x) ⊙ (w3x) | sigmoid = differential pair (native); ⊙ = **Gilbert/translinear multiplier** per unit | textbook standard cells | transpose of ⊙ = multiply by the partner signal — cheap | **E-tier w1: 10% gain error ΔCE +0.006, cos_clean 0.945 — FREE** at translinear-practice tolerances |
| 7 | RMSNorm ×2 per block (norm-after-sublayer) | bound each sublayer output | divisive normalization (as #4), **no mean-subtraction path** (cheaper than LayerNorm) | same as #4 | symmetric Jacobian | as #4; ~4L+1 normalizers total = the largest NEW periphery count |
| 8 | Residual stream | z + branch outputs | **current-summing bus (KCL node)** | wires | pass-through | norm-after bounds every injection — the arch change is itself the tolerance fix |
| 9 | Final RMSNorm | bound pre-readout state | as #4 | as #4 | symmetric | bounds the **ADC** dynamic range at the digital boundary |