diff options
Diffstat (limited to 'docs/campaign/CASCADE_ABLATION_PLAN.md')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 20 |
1 files changed, 20 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index ac602cf..b456ccd 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -464,3 +464,23 @@ UNTOUCHED (the control measuring erosion damage vs BP twins); PARALLEL bf1e3 con from ckpt-40000 on the farm (floor 1e-3 for the remaining 18.8k steps). Endpoint comparison becomes a clean quad: EP-control(3e-4) / EP-floor-lift(1e-3 from 40k) / BP-s1 / BP-s2 — quantifies BOTH the erosion damage AND the fix's recovery in one shot. + +### RESULT 10 (2026-07-10 19:0x): EPOCH ENDPOINTS + "NENG KAN" GATE PASSED + stage1b (improved recipe) launched. +**Epoch endpoints (58,800 steps / 361M tokens, OLMo2, AdamW):** +| arm | best val CE | +|---|---| +| BP s1 / s2 | **1.2750 / 1.2509** | +| EP (floor 3e-4 fixed) | **1.4802** (zero guard events end-to-end) | +| EP bf1e3-cont (floor->1e-3 @40k) | 1.4835@46k, running to 58.8k | +**EP-BP gap at epoch scale = +0.22** (was +0.03 at 4k): the late-SNR cos erosion (1.0 -> ~0.92-0.95) +is a REAL, horizon-growing CE cost with fixed floor 3e-4. Mechanism + dial both established +(dose-response); the improved recipe is designed to close this. +**GENERATION GATE ("neng kan") PASSED:** casc_gen.py (new; plain-forward standard-LLM inference) from +EP s55000: coherent multi-paragraph TinyStories — named characters, balanced-quote dialogue, +cause-effect, emotional arc (minor charm-defects vs BP's tighter coherence, consistent with +0.22). +**A 42.75M standard 12-layer transformer trained end-to-end WITHOUT backprop tells coherent stories; +inference is a plain forward pass.** Task #15 demo artifact exists. +**stage1b launched (the improved-recipe head-to-head):** stage1b_ep_muon (local GPU1: Muon + floor +3e-4 + bf_late 1e-3@15k + kretry + cosine[now also on Muon via build_hybrid total_steps]) vs +stage1b_bp_muon (farm GPU6: Muon + cosine). Expectation: EP ~1.25-1.35 (Muon -0.13 and erosion fix +~-0.1+), BP+Muon anchor moves too. ~8h both. |