diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 20:56:01 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 20:56:01 -0500 |
| commit | 6d9fcd748df0836528b31f29717ac3bc8715d1dd (patch) | |
| tree | 288abf1e70f7d955f7d5d008d9648f3053986a00 /docs/campaign | |
| parent | 196ee7d8121fccaae630e408bf29f6664ff351f5 (diff) | |
RESULT 10: epoch endpoints (BP 1.2509/1.2750 vs EP 1.4802, +0.22 erosion cost); GENERATION GATE PASSED (casc_gen.py, coherent stories, no-backprop 42.75M); stage1b improved-recipe pair launched (Muon cosine via build_hybrid)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs/campaign')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 20 |
1 files changed, 20 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index ac602cf..b456ccd 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -464,3 +464,23 @@ UNTOUCHED (the control measuring erosion damage vs BP twins); PARALLEL bf1e3 con from ckpt-40000 on the farm (floor 1e-3 for the remaining 18.8k steps). Endpoint comparison becomes a clean quad: EP-control(3e-4) / EP-floor-lift(1e-3 from 40k) / BP-s1 / BP-s2 — quantifies BOTH the erosion damage AND the fix's recovery in one shot. + +### RESULT 10 (2026-07-10 19:0x): EPOCH ENDPOINTS + "NENG KAN" GATE PASSED + stage1b (improved recipe) launched. +**Epoch endpoints (58,800 steps / 361M tokens, OLMo2, AdamW):** +| arm | best val CE | +|---|---| +| BP s1 / s2 | **1.2750 / 1.2509** | +| EP (floor 3e-4 fixed) | **1.4802** (zero guard events end-to-end) | +| EP bf1e3-cont (floor->1e-3 @40k) | 1.4835@46k, running to 58.8k | +**EP-BP gap at epoch scale = +0.22** (was +0.03 at 4k): the late-SNR cos erosion (1.0 -> ~0.92-0.95) +is a REAL, horizon-growing CE cost with fixed floor 3e-4. Mechanism + dial both established +(dose-response); the improved recipe is designed to close this. +**GENERATION GATE ("neng kan") PASSED:** casc_gen.py (new; plain-forward standard-LLM inference) from +EP s55000: coherent multi-paragraph TinyStories — named characters, balanced-quote dialogue, +cause-effect, emotional arc (minor charm-defects vs BP's tighter coherence, consistent with +0.22). +**A 42.75M standard 12-layer transformer trained end-to-end WITHOUT backprop tells coherent stories; +inference is a plain forward pass.** Task #15 demo artifact exists. +**stage1b launched (the improved-recipe head-to-head):** stage1b_ep_muon (local GPU1: Muon + floor +3e-4 + bf_late 1e-3@15k + kretry + cosine[now also on Muon via build_hybrid total_steps]) vs +stage1b_bp_muon (farm GPU6: Muon + cosine). Expectation: EP ~1.25-1.35 (Muon -0.13 and erosion fix +~-0.1+), BP+Muon anchor moves too. ~8h both. |