diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-14 22:37:07 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-14 22:37:07 -0500 |
| commit | 88840bfbd69d6a565786100a6744b7646c632e72 (patch) | |
| tree | f8cc95a2064586d0b1f432f217ec06ed0c6f10bd /docs/campaign | |
| parent | 392d0ec28ebaa9d5fd77e16094638857e3ddacd7 (diff) | |
RESULT 24: fw72m_cent pre-registration (centered crown rerun, bcap-v3 0.9, gradtest 0.9999877 gate passed, chained behind ladder)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs/campaign')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 15 |
1 files changed, 15 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 8e9faf3..c0044ee 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -568,6 +568,21 @@ direction), not training-under-fault; wave-2 = co-training with faults injected +### RESULT 24 (2026-07-14): fw72m_cent LAUNCH — window-aware centered crown rerun (pre-registered). +User call: "72m从头跑centered试试看". From-scratch 234k-step FineWeb rerun applying RESULT 22: +original fw72m flags EXCEPT --est centered, --bf_late 3e-3 --bf_late_at 20000 (ride the arms-winner +beta instead of 1e-3@60k), --beta_cap_rho 0.9 (v3 threshold: attack only near true divergence; +v2's 0.7 conflated slow contraction and starved beta to 2e-5 in the c2 segment). Early phase keeps +the stock ramp + 3e-4 floor (the early beta dip is a sigma-transient stabilizer, not a bias fix). +Correctness gate BEFORE launch: ddp_gradtest with centered+amp — cos(DDP 2-rank, single-GPU +big-batch) = 0.999987714, relerr 5.0e-3 (bf16 band; fp32 harness was 0.999999999). Chained behind +the gap-scaling ladder on GPU1+3 (launcher polls GS markers). Cost: centered ~1.8x step time. +Predictions: (a) no 195k-style blow — beta_t = min(3e-3, ceiling(t)) tracked by bcap instead of a +fixed floor crossing the falling ceiling; (b) best val beats 3.7117 (bias down one order at matched +loop gain + tail SNR up); (c) honest gap vs BP twin 3.2884 lands ~0.25-0.35 (registered guess). +Failure mode to watch: bcap-0.9 rides too close to the edge -> guard storms (kretry/drift) without +progress; lever = drop threshold toward 0.8, resume from last 5k ckpt. + ### RESULT 23 (2026-07-14): GAP-SCALING SUITE — PRE-REGISTRATION (launched, results pending). Question: how does the EP-BP epoch gap scale with model width under the FROZEN stage1b recipe? Design: C ∈ {128, 192, 256, 384} x L12 H8 T256 B24, tinystories_bpe, full data-matched epoch |