summaryrefslogtreecommitdiff
path: root/EMAIL_BEN_DRAFT2.md
diff options
context:
space:
mode:
authorYuren Hao <yurenh2@illinois.edu>2026-07-12 05:44:18 -0500
committerYuren Hao <yurenh2@illinois.edu>2026-07-12 05:44:18 -0500
commit411870d64f00181ea41a8059f4b53f5b232447c0 (patch)
tree391b22448b2c2398e7857e3c2af39abc9cd037c2 /EMAIL_BEN_DRAFT2.md
parent155478b7ffdd9015fde13884b1ebff9b92148a20 (diff)
E-tier wave-1 ledger (RESULT 14 + HW map rows) + cost model v2 recalibrated on measured H200 datapoint
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'EMAIL_BEN_DRAFT2.md')
-rw-r--r--EMAIL_BEN_DRAFT2.md19
1 files changed, 11 insertions, 8 deletions
diff --git a/EMAIL_BEN_DRAFT2.md b/EMAIL_BEN_DRAFT2.md
index f367b12..0b6fc4a 100644
--- a/EMAIL_BEN_DRAFT2.md
+++ b/EMAIL_BEN_DRAFT2.md
@@ -16,17 +16,17 @@ The results promised yesterday, concretely:
- **The gap ledger.** At matched hyperparameters, 4k steps: EP statistically indistinguishable from its tuned backprop control (three seeds each). Full epoch: 0.050 nats behind, with the mechanism identified — the late-training decline of estimator signal-to-noise as the true gradient shrinks against a fixed precision floor (the quantified version of your β remark) — and a β-schedule that removes most of it. Muon's improvement over AdamW transfers to EP gradients at full magnitude.
- **Solver cost.** Our nudged phase runs the full EP step at 3.2× the FLOPs of a backprop step (measured; single-sided, K=3 full-depth sweeps per nudge). For calibration against your Appendix B: that ratio is what turns an 18-day run into a ~4-day one at equal hardware.
-On costs, anchored on measured throughput (TF32 verified bit-exact):
+On costs — calibrated against a measured datapoint rather than list pricing (a 1B × 10B-token backprop run we executed on rented H200s cost ≈$1k all-in), combined with the measured 3.2× EP/BP FLOP ratio:
-| run | cost, incl. 1.4× failure allowance |
+| run | cost at market H100/H200 rates, incl. 1.3× failure allowance |
|---|---|
-| 300M × 6B tokens — recipe validation on a standard corpus | ~$1k |
-| **1B × 20B tokens, Chinchilla-optimal** | **~$12k** |
-| 1B backprop control | ~$4k |
+| 300M × 6B tokens — recipe validation on a standard corpus | ~$400 |
+| **1B × 20B tokens, Chinchilla-optimal** | **~$4k** (≤$8k even at the uncalibrated worst case) |
+| 1B backprop control, same tokens | ~$1.5k |
-Within $50k this leaves room for a second seed or a first 1.5B segment. The allowance is modest because the failure modes are characterized rather than assumed, and training checkpoints every 5k steps — a failure loses only the segment since the last checkpoint. A one-day calibration on your instances (~$500) would reduce the uncertainty from ±30% to ±10–15%; we would be glad to run that first.
+Three qualifications. The numbers assume the TF32 path we have verified bit-exact against fp32; a standard mixed-precision variant (bf16 matrix products, fp32 states and energy accumulation) is not yet validated for EP and would roughly halve them if it passes our gradient-fidelity gate. AWS list pricing runs 1.5–2.5× market rental per GPU-hour, with spot or capacity-block reservations closing most of that gap. And checkpoints every 5k steps bound any failure to the segment since the last one. A one-day calibration on your instances (~$500) would replace these estimates with measurements; we would be glad to run that first.
-The question we are weighing beyond that: **whether 3B at full Chinchilla, or 7–8B at reduced token count, is feasible** (~$25–30k and ~$50k respectively at current measured throughput). We are not requesting this now — the intention is to complete the runs above and revisit the larger scale on the basis of those results.
+This pricing changes the scale question. Within $50k, the 1B milestone and its control leave room for one of: **3B at full Chinchilla scaling (~$30–40k at market rates), or 7–8B at a GPT-3-style token budget (~$25–35k)** — AWS on-demand pricing adds roughly half on top, and the mixed-precision variant, if it validates, subtracts it back. Our intention is unchanged — complete the 1B result first and let it argue for the larger run — but the envelope does reach one of the two.
Separately, and possibly of direct interest to Rain: your paper leaves open whether F_PCN can be realized efficiently in hardware — **that question is the one our recipe was designed around.** Every trained operation was chosen to have a known analog implementation: divisive (current-mode) normalization, fixed rotations for the position encoding, translinear multipliers for the gated MLP, the subthreshold-exponential/KCL circuit for attention's softmax, and EP's two relaxation phases for the learning rule. A component-by-component hardware mapping is written up; we can share it.
@@ -39,5 +39,8 @@ Yuren
## 发送前 checklist(内部,不进邮件)
1. [LINK] 换成真实 Overleaf 链接(zip 已备好: overleaf_dynamics_v3.zip,含两对象改写+KHS26引用+floss重调数字)
2. 附 a1_demo.ipynb(token 已内置)
-3. gap 数字核对: f3e3cont 终值若 <1.2808 → "0.050 nats" 段落改为新 gap + "a β-schedule removes most of it" 升级为具体数字
+3. ~~gap 数字核对~~ RESOLVED: f3e3cont 终值 1.2883 > 1.2808 → "0.050 nats" 保持不变
4. 3.2× 的对照句("18-day → ~4-day")口吻是否合适你再定——它来自他们自己的 Appendix B,准确但直接
+5. 成本表 2026-07-12 已按用户实测锚点重校准(1B×10B BP H200 租赁 ≈$1k → $16.7k/EF 全包上限;
+ 市场价 H100/H200 TF32 路径 $6-9k/EF)。旧表锚 AWS-A100-savings,每 FLOP 贵 2.5-3×,已废。
+ 完整推导见 docs/campaign/COST_MODEL.md