path: root/docs/hardware/MSCALE_SUPPLY_CHAIN_SEARCH.md

# M-SCALE EP TRAINER — FINAL SYNTHESIS (chief architect, 2026-07-13)
All numbers below are the **hostile-audit-corrected** figures, not the proposals' headlines. Bands are §4 all-in (loaded labor included) unless marked "cash/parts."

---

## 1. Executive summary

- **The null hypothesis is not merely beaten — it is DEMOLISHED FROM BELOW.** The partner-CIM path the null rests on is weaker than stated: Shanbhag's own Nov-2025 slides label the 28nm ESSERC chip a **Digital IMC** ("AIMCs are hard to scale → digital IMCs"), and the 65nm DIMA trainer die's published record (JSSC-2018) shows mode (ii) is **verifiably absent** — outputs are 3 comparators plus a 6b ADC pair "for testing purposes." The null's "grade-B conditional on two unverified die modes" is, on the published evidence, grade-C/zero. The two die-mode questions are no longer open; they are answered NO for every documented UIUC die.
- **Consequence: any surviving unconditional B−(M) purchasable path beats the null on grade AND gating.** Two exist. Neither beats $20–40k on loaded cost; both beat it on conditionality, purchasability, and mode evidence.
- **Winner: T64** — one 64×64 time-multiplexed TLC7528 MDAC tile, word-streamed weights (R=610). Word-level reload makes **mode (i) transpose exact by construction** (same physical cells, transposed word — the strongest mode-(i) evidence found on any substrate, better than any die doc). Corrected: parts $17–22k, loaded $52–82k, 100M-token TinyStories in 9–14 d PASS (pipelined θ-read), grade B−(M) contested ~60/40, purchasable today (TLC7528CDW: 845 @ DigiKey $5.05@1k, ACTIVE, 9-wk lead — buy-ahead mandatory).
- **The grade ceiling is structural, not financial:** at M scale on purchasable hardware, sub-block residency forces digitized inter-block activations → **B/B− is the ceiling** (claim-engineering axis's residency theorem, confirmed independently by five audits). B+(M) requires a block-resident ≥12d² ≈ 2×10⁵-cell integrated array at c_cell ≤ $10⁻² — i.e., partner-class or self-owned silicon. Grade A(M) exists nowhere (M-cell analog store = refresh machine; LF398 droop arithmetic kills it at every audit).
- **Cheapest credible path per band:** **≤$3k** — no M-trainer exists under all-in accounting; the correct spend is the metrology package: $0 GPU fault injections (activation-requant, 2–6% transpose asymmetry, contrast-SNR ledger) + ~$500 single-column TLC7528 settle/ENOB rig + ~$500 LCD transpose-fidelity rig. These gates retire the top risks of every $50k+ line. **≤$30k** — T64 parts ($17–22k) with self-labor (cash accounting only; no loaded-band build fits). **≤$150k** — T64 loaded ($52–82k), with T128 upgrade (~$75–95k) if a 1B-token claim is required.
- **Every flash/memristor/printed substrate is DEAD by physics, not price:** gate-input NOR flash forbids transpose MVMs and bitline sums erase per-cell contrast (analog-CIM axis); 10⁴–10⁵ endurance vs ~10⁶–10⁸ required writes (§2.2); printed arrays fail the 10× capacity kill rule by 2,400×. Money cannot fix any of these.

## 2. The cost frontier — surviving combined designs (audit-corrected)

| Design (axes combined) | N_phys | R | c_cell (source) | Audited total $ | Grade @M | Wall-clock 100M / 1B tok | TTFR (θ-read / full run) | Fragility |
|---|---|---|---|---|---|---|---|---|
| **T64** (reuse × salvage-priced MDACs) | 4,096 | 610 (M=2.5M) | $4–5 all-in (TLC7528CDW $5.05@1k DigiKey; gray AD7528JN $0.68 UTSource halves the line if screened) | parts $17–22k; loaded **$52–82k** | **B−(M)**, modes (i) exact by word-transpose, (ii) tile-edge CDS contrast; contested 60/40 | **9–14 d PASS** (pipelined) / DEAD (T128: 7–18 d, +$40k) | 6–9 wk / ~6–8 mo | AMBER-RED: TI sole active mfr, 845 in stock vs 2,250 needed; 110% buy-ahead + AD7628KR footprint |
| **STREAM-128** (claim-eng × digipot, SPI-redesigned) | 16,384 | 72 (M=1.18M) | $1.0–1.5 (MCP4351 class) | parts $17–24k; loaded **$80–120k** | B−(M), unconditional modes | 14–21 d MARGINAL / DEAD | 5–8 wk / 6–9 mo | YELLOW: Microchip single-mfr, huge reel stock |
| **PRINTBAR** (consumer LCD × reuse) — conditional on mode-(ii) repair | 5.9×10⁵ | 4.2 | $1.9×10⁻⁴ (ELEGOO 12K panel $109.99, ChiTu) | parts $6–12k; loaded **$30–60k** | B− *only if* mode-(ii) resurrection lands; else C | 7–16 d PASS-MARG (14–32 d honest path) / DEAD w/o 4 arms (+$8–15k) | **3–6 wk ($500 rig)** / 4–6 mo | GREEN panels; **RED driver board — no COTS HDMI bridge for the 51-pin 12K panel exists; uncosted critical path** |
| **EPT-48** (free silicon × reuse; successor line, not a 2026 build) | 2,304 | 1,085 | cash $0.5–0.7 / loaded $15–43 | cash $1.5–3.5k (slot-conditional); loaded **$50–120k** | B−(M); silent-C failure mode if contrast cell underdelivers | 5–9 d PASS / 13–25 d multi-die | ~12–15 mo (TT) / **24–36 mo** | Discretionary free-slot lottery; 2-runs/yr cadence; unproven contrast cell |
| **SC-1M** (Shenzhen 1T1C × R=L) — only with the $8–15k analog product engine | 49,152 | 20 (M=1M) | $0.033–0.05 (2N7002 $0.0081 LCSC + JLC joints) | **$40–78k** | B− conditional on product engine; as-budgeted BOM = C/DEAD (143 d gradient-read wall) | 3–7 d @10 µs settle / 21–40 d MARG | 4–7 wk ($300–500) / 5–8 mo | GREEN parts; the DAQ engine is the fragile line |
| DEAD: analog-flash CIM (physics), printed/electrochem (2,400× capacity), salvage GHOST-DAC as trainer (grade C; lives as $3–6k testbed), U1 65nm DIMA (mode-ii absent → C), U0 28nm (digital IMC) | | | | | | | | |

**Key frontier fact:** R-engineering is the axis that made M-scale purchasable at all — R=610 converts what would be $8–15M of [D]-priced discrete cells into $10–22k of MDACs, and volatile cells are *forced* (each physical cell absorbs ~10⁸ writes/run, killing every nonvolatile chemistry), which is why MDAC/digipot/SRAM registers win everywhere.

## 3. Recommended program

**Stage 0 — this week, $0.**
1. GPU fault injections (existing wq8/wq6 harness): (a) boundary-activation requantization at 12/14/16b; (b) forward/transpose gain asymmetry at 0.5/2/6%; (c) contrast-read additive noise at the AD7606B ENOB point; (d) 1-LSB stall @8b with 24b shadow. Any §3 floor missed without costed mitigation kills T64 before purchase.
2. One GPU-week theory question (SC-1M dissent): does the cascade proof extend to sub-block settling (attention/MLP settled separately)? A YES doubles-to-quadruples R program-wide and opens the $3k band at M=1M.
3. Emails: (a) **openpdk@ihp.de** — "Is a free SG13G2 open-source slot confirmed for 2026/2027, and how many dice ship?" (b) **UIUC/Shanbhag, amended ask, verbatim:** "We've read the JSSC-2018 die docs — the 512×256 bank's outputs are comparator decisions plus the 6b test ADC pair, and the 28nm DiT is digital IMC per your Nov-2025 deck. Two questions: does ANY die variant in the group's inventory expose per-column analog outputs before aggregation, and would you co-design a per-column two-phase contrast-readout macro on a future run?" (c) Calendar: TetraMem MLX200 EVK (2H-2026), EnCharge EN100 Rd-2 waitlist.

**Stage 1 — weeks 1–6, ~$1.2–2.5k (self-funded band).**
- **Buy:** 4× TLC7528CDW (DigiKey, $8.48@1), 1× AD7606BSTZ ($47.03, 3,620 stock), LF398MX/NOPB ×8 ($1.91), 1 ELEGOO Saturn 12K panel ($109.99, ChiTu) + RPi Global Shutter cam ($50), PYNQ-Z2 (~$179–235, Seeed/DFRobot — LCSC is OOS), JLC probe PCBs.
- **Measure (pre-registered kill criteria):** (a) single-column 64-load TLC7528 system settle — **kill T64 if >7 µs**; commit if ≤3 µs; (b) ENOB after per-cell digital pre-scale cal under 1-hr drift — kill if <7.0; (c) LCD rig: transpose reciprocity asymmetry (kill threshold 3%, fault-injected) and effective bits at 400:1 contrast; (d) contrast-null offset vs the 0.1–1 mV signal ([D] lesson, measured not argued).

**Gate G1 (week 6):** settle+ENOB pass → Stage 2. Settle fails but LCD passes → pivot resources to the PRINTBAR mode-(ii) repair question. Both fail → the honest answer is that no purchasable M-trainer exists in 2026 and the program is the rung-A tile + successor silicon.

**Stage 2 — months 2–8, T64 build ($17–22k parts; grant application for $52–82k loaded).**
- Buy-ahead: 2,250× TLC7528CDW at 110% (~$11.4k) the day G1 passes; qualify AD7628KR footprint as alternate; screen 100 pcs incoming.
- Corrected architecture (per audit): offset-binary codes + digital ½·Σx subtraction (NOT the Figure-4 four-quadrant circuit); ADC pipelined under next-settle (else 15–20 d MARGINAL); loads from Zynq DDR (FRAM = nonvolatile mirror only); FMC-class carrier for 16 byte-lanes; auto-zero chopper TIAs.
- **Pre-register the claim sentence before gateware starts** (see §5).

**Stage 3 — parallel lottery tickets, ~$570.** TT rehearsal die (triode cell + contrast sampler, 2×2 analog tiles) on the next open shuttle (~Q4'26): measures the one circuit that gates the EPT-48 successor. Zero coupling to Stage-2 schedule.

## 4. Supply-chain plays ranked by leverage (≥10× only)

1. **Reuse factor R itself (500–1000×):** word-streaming one physical tile converts cell price into wall-clock; it also deletes the dual-copy Wᵀ drift channel for free. Every dollar spent raising R beats any component substitution.
2. **Consumer-volume converter banks (10–30× on periphery):** PCM1808 audio ADC $0.90/ch/24b, TLC5947 LED-driver DAC $0.17/ch vs $3–10/ch instrumentation. Axis-independent — applies to T64, STREAM-128, and any partner harness. The restore-path line, which the spec flags as "where the money goes," collapses.
3. **Gray-market MDACs (7–12× on the largest BOM line):** AD7528JN $0.68–0.72 (UTSource, used pulls, RFQ depth) vs TLC7528 $5.05@1k new. Requires 100-pc ATE screen + 15% overage; TI-ACTIVE new stock caps downside at 7.4×.
4. **Free-silicon channels (≥20× on fab, paid in calendar):** IHP $0 vs €73k direct — but 24–36 mo and lottery-gated. A quote, not a plan, until the openpdk email returns.
5. **Consumer LCD panels (10³–10⁴× on c_cell):** $1.9×10⁻⁴/cell — worthless until mode (ii) is repaired; then decisive.
Component-level optimizations (chopper choice, FRAM vs SRAM, PCB vendor) all move ≤2–3× — stop spending search effort there.

## 5. Claim ruling

**Minimum object licensing "an LM trained in analog hardware" at a flagship venue: dilution level D2 / grade B(M)-to-B−(M)** — both equilibration phases physically settled on the substrate, states digitized at the tile edge, update composed digitally from measured contrasts. Precedents that survived review with exactly this shape: Yi et al., Nat. Electron. 2022 (64×64 activity-difference training); Momeni et al., Science 382:1297 (2023). Anything weaker (digital nudged phase, analog-forward-only) is the [C] forbidden object; every audit independently converged on the same one-sentence referee kill: *"digital training with an analog gradient estimator."*

**Pre-registration package (before any board order):** (1) the exact sentence — for T64: *"A 2.5M-parameter transformer LM in which every trainable weight's free and nudged MVMs, forward and transpose, are executed by physical analog settles on one time-multiplexed crossbar, and every gradient originates as a measured two-phase contrast at the tile edge; weight state, nonlinearities, rank-1 accumulation, and activation logistics are digital, declared"* — with a concessions table; (2) anti-C evidence: logged I/O traces proving no digital forward/Jacobian exists in the loop + the analog-oracle ablation (updates from measured states vs digital-twin states); (3) the fault-injection ledger; (4) venue Nat. Electron./NeurIPS tier, mechanism-framing ("trained BY physical equilibrium dynamics"), never "fully analog." The audit's warning stands: the drafted T64 sentence must **not** omit digital nonlinearities and digital rank-1 — concealing exactly the facts that trigger auto-reject is the failure mode.

## 6. What this changes about the Dillavou and UIUC conversations

**UIUC:** the conversation inverts. We are no longer asking them to verify two die modes — the published record answers NO (65nm: comparators + test-ADC only; 28nm: digital IMC by their own slides). The new asks: (a) inventory check — any undocumented die variant with per-column analog readout; (b) a co-designed contrast-readout macro on a future run (that artifact, and only that, would restore a B+(M) partner path and the $10⁻³–10⁻² c_cell frontier); (c) meanwhile UIUC's real value is harness/gateware collaboration and the digital-IMC baseline — the T64 gateware ports unchanged to any future analog macro. The $20–40k "conditional-B" line should be retired from our internal planning documents.

**Dillavou/Penn:** unchanged as the grade-A physics rung (P1 at M scale is DEAD by 2–4 orders on swap-settle — years of gate-cap programming). The outreach framing sharpens into the two-paper structure: grade-A primitive at 10¹–10² edges (their apparatus + our rung-A tile) and a separate B−(M) M-scale LM (T64). We can offer them something concrete: our Stage-1 rigs measure effective-bits-under-limit-cycle — the exact open question in their 2505.22887 — on COTS hardware in six weeks. For Scellier specifically, the pre-registered wording is the deliverable: he is the referee class that polices the C-boundary, and showing him the concessions table *before* building converts the harshest reviewer into a co-author-shaped ally (consistent with the standing two-stage outreach plan).

## 7. Dissents and resurrection conditions worth preserving

1. **PRINTBAR mode-(ii):** demonstrate the LC panel as the quasi-static factor in a Wang-style integrating dot-product at ≥1 kHz effective token rate (segment-scanned partial updates), or find a COTS per-pixel multiplying/log-mode sensor <$200 → restores B− at $6–12k parts and 4–8 d runs. Testable on the Stage-1 $500 rig.
2. **Sub-block settling theory (SC-1M dissent):** if attention and MLP settle separately under the cascade proof, R jumps to 2L–4L program-wide; $3k band opens at M=1M. One GPU-week, $0 — highest information-per-dollar item in the program.
3. **EPT-48:** written IHP confirmation of a 2026/27 free SG13G2 slot PLUS a measured TT contrast cell with <1 mV null offset over 5 ms hold → strongest self-owned line; otherwise it stays a $570 lottery ticket.
4. **TetraMem MLX200 EVK (2H-2026):** if per-cell analog program/read is exposed, it is the only merchant substrate where B+(M) is physically conceivable. Calendar it.
5. **EnCharge EN100 Rd-2:** any transpose or raw-array mode in the EVK docs → jumps to the leading M-scale line (unlimited-endurance analog CIM, merchant track).
6. **Digipot 7-ENOB-under-drift:** if per-cell cal is demonstrated to hold (the CLLN question, answerable on the P32 probe), a ~$2.6–4.5k M=1M machine exists and T64's cell line drops 7×.
7. **Row-serial revival:** a system-level 250 ns 8-bit analog MAC row settle, or a Chinchilla-25M token budget for M=1M, revives R≈10⁴ architectures (8 d even at 5 µs settle).
8. **Mythic fire-sale watch:** 76.8M analog cells at salvage prices rewrites the cost algebra overnight — but only with an opened program/read API and ≥10⁸ endurance, both currently absent; grade capped C by gate-input physics regardless.

**Files:** spec `/home/yurenh2/ept/docs/hardware/MSCALE_COST_ALGEBRA_SEARCH_SPEC.md`; prior audits `/home/yurenh2/ept/docs/hardware/HW_FIRST_PRINCIPLES_SEARCH.md`, `COMPONENT_HW_MAP.md`, `/home/yurenh2/ept/CLOCKLESS_ANALOG_MVP_PLAN.md`. Key external anchors (live 2026-07-13, from the audited axis reports): TLC7528CDW $5.05@1k/845 stock DigiKey; AD7606BSTZ $47.03/3,620; AD7528JN $0.68 UTSource (used); MCP4351 $1.41@2.5k Mouser; ELEGOO 12K panel $109.99 ChiTu; PCM1808 $1.81, TLC5947 $4.17 DigiKey; 2N7002 $0.0081 LCSC; PYNQ-Z2 ~$179–235 (LCSC OOS); Shanbhag Nov-2025 deck (publish.illinois.edu) + JSSC-2018 author PDF (shanbhag.ece.illinois.edu); Yi et al. Nat. Electron. 2022; Momeni et al. Science 2023.

---
*Generated by a 20-agent adversarial workflow (cost-algebra -> 9 leverage axes x hostile audits -> synthesis), 2026-07-13. Raw: workflows/wf_7f2db7fc-ef5.*