summaryrefslogtreecommitdiff
path: root/docs/campaign
diff options
context:
space:
mode:
authorYuren Hao <yurenh2@illinois.edu>2026-07-10 03:05:42 -0500
committerYuren Hao <yurenh2@illinois.edu>2026-07-10 03:05:42 -0500
commit129aee2fd1fb344c1980ec687d4bff3f9e295734 (patch)
treed7e975adc202f35cc3bee709bc7223c8955f567f /docs/campaign
parent3d1bae3cd3e3333c2134669d2453fbb9d6a1adf4 (diff)
RESULT 3: K4 depth-parity SEALED EP-favorable (EP 1.8907 <= BP 1.9192 @ L12xC512); full-epoch generation run auto-launched (6.9h)
Diffstat (limited to 'docs/campaign')
-rw-r--r--docs/campaign/CASCADE_ABLATION_PLAN.md14
1 files changed, 14 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md
index a4fab80..279b2fe 100644
--- a/docs/campaign/CASCADE_ABLATION_PLAN.md
+++ b/docs/campaign/CASCADE_ABLATION_PLAN.md
@@ -217,3 +217,17 @@ E-tier: after C1 lands (tolerance scripts port directly)
Naming: `casc_*` runs, wandb project **ept-cascade**. Gates report mean over ≥4 batches.
In-flight single-block arms (rescv2, govfloor, fastfull/fastpair, gov_s11-14) continue untouched —
they carry the dynamics paper + the two-stage-recipe science; D1 takes over the DEMO role only.
+
+### RESULT 3 (2026-07-10 03:03): K4 DEPTH-PARITY SEALED (EP-favorable) + full-epoch launched.
+- **beta-floor 3e-4 EP 3-seed: 1.9005 / 1.9125 / 1.8591 = MEAN 1.8907** vs BP 1.9169/1.9194/1.9214
+ (1.9192). **EP <= BP at L12xC512 (real GPT-small shape)** -- all 3 EP seeds below the best BP seed,
+ cos pinned 1.0000 throughout, zero skips. The L12 depth-tax is FULLY removed by the beta-floor; K4
+ closes EP-favorable. (Un-floored control was 2.00 + unstable/non-reproducible -- see RESULT 2.)
+- Headline now: "standard L12 transformer, no backprop, equilibrium-EP with beta-floor = BP quality
+ (slightly better) at matched tuning, real GPT-small shape."
+- **FULL-EPOCH run LAUNCHED (user directive, auto-launched on verdict):** epoch_ep_bf3e4 -- 58,800
+ steps = 1 epoch over TinyStories-BPE (361M tokens), beta_floor 3e-4 + --cosine (new flag), warmup
+ 500, save_every 5000. Running 2.376 it/s solo on GPU1 -> ~6.9 h. This is the "neng kan" generation
+ demo (task #15). BP twin epoch DEFERRED (no free GPU; parity already sealed so it is nice-to-have).
+- Next: generation samples at checkpoints; BP-twin epoch when a GPU frees; then scale-up corpus
+ decision (FineWeb-Edu vs OLMo2/Dolma) for the larger model.