1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
|
# Cost model v2.1 (2026-07-12) — sourced July-2026 prices + measured amp speedup
Supersedes v2 (same day; fixed a 1000× unit-label error — exaFLOP = 1e18, so the user's
anchor is $16.7/EF not "$16.7k/EF"; all dollar TOTALS in v2 were computed as ratios and
were correct) and v1 (anchored on AWS-A100-savings, ~2.5-3× too expensive per FLOP).
## Sourced prices (July 2026)
| item | price | source |
|---|---|---|
| Market H100 | $1.87–2.99/GPU·h | Vast $1.87, RunPod PCIe $1.99 / SXM $2.69, Lambda $2.99 (intuitionlabs.ai/articles/h100-rental-prices-cloud-comparison, runpod.io/pricing, lambda.ai/pricing) |
| Market H200 | $2.30–4.39/GPU·h, median ~$4 | same survey; FluidStack low end |
| **AWS p5e (H200) capacity blocks** | **$39.80/instance·h = $4.97/GPU·h** (Ohio; US-West $6.22) | aws.amazon.com/ec2/capacityblocks/pricing — **+20% hike effective 2026-07-01** (p5en $5.20) |
| AWS/marketplace ratio | **~2–2.5×** | above (v2's "1.5–2.5×, blocks close the gap" is stale — blocks got hiked) |
H100 and H200 have identical compute (H200 = more/faster HBM); for 1–7B models rent H100
at $2–2.7 — strictly better $/FLOP than H200.
## Rates (our stack, measured)
- Anchor (empirical ceiling): user's real run, 1B×10B BP on rented H200 ≈ $1k = 60 EF → **$16.7/EF**
all-in with NO stack optimization (implied util 4–6% of bf16 peak). Kept as the worst case.
- Utilization: measured ~19% of TF32 peak on A6000 eager → H100 TF32 (494 TF) at 15–25% =
75–125 TF/s eff → **TF32 rate: $6–9/EF market**.
- **amp (bf16 autocast, fp32 states/d/E): VALIDATED 2026-07-12** — gate cos 0.9682 vs fp32 0.9687;
3-seed 4k A/B mean 1.7406 vs fp32 1.7314 (inside seed-noise band, amp_s1 beat the BP mean);
**measured 1.56× wall-clock at C512/A6000** (2.785 vs 1.789 it/s solo). Conservatively apply
1.5× (tensor-core-bound share grows with model width, so this is a floor at 1–3B):
**amp rate: $4–6/EF market; $9–14/EF AWS blocks**. Full-epoch confirm (stage1b_amp) running.
- EP/BP FLOP ratio 3.2× (measured); EP FLOPs = 19.2·N·D. Failure allowance 1.3× (ckpt every 5k).
## Table (amp rates; buffered ×1.3)
| run | EP FLOPs | market | AWS blocks |
|---|---|---|---|
| 300M × 6B (recipe validation) | 34.6 EF | **~$300** | ~$500 |
| **1B × 20B Chinchilla EP** | 384 EF | **~$2.5k** | **~$5k** |
| 1B × 20B BP control | 120 EF (6ND) | ~$700 | ~$1.5k |
| **3B × 60B full-Chinchilla EP** | 3,456 EF | **~$18–27k** | **~$35–48k** |
| **7B × 20B (GPT-3-style) EP** | 2,688 EF | **~$14–21k** | **~$27–38k** |
| 7B × 140B full-Chinchilla EP | 18,816 EF | ~$100–150k | ~$190–260k |
TF32 fallback (if stage1b_amp fails at epoch scale): multiply by ~1.5.
## Consequences
1. **With amp landed, BOTH large options fit the $50k AWS envelope individually**
(3B-Chinchilla ~$40k, 7B×20B ~$30k, each incl. the 1B milestone ~$6.5k); at market rates
either fits with wide margin — quote AWS to Ben (his credits), note market is ~2× cheaper.
2. The calibration day (~$500 on his instances) resolves the util spread (15–25%) → ±15%.
3. bf16-amp moved from "upside" to "measured" — the ÷1.5 is in the table, not a promise.
Residual upside: compile/flash-attn in the relax loop (not priced), util growth at width.
|
