RESULT 4: QK-norm validated parity-preserving (EP 1.8868 <= BP 1.9066); qk_norm complements but does NOT replace beta-floor (sig_tok still grows); Stage 1 TinyStories epoch launched

1 files changed, 16 insertions, 0 deletions

diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md
index 585ecc1..8e0ac43 100644
--- a/docs/campaign/CASCADE_ABLATION_PLAN.md
+++ b/docs/campaign/CASCADE_ABLATION_PLAN.md
@@ -267,3 +267,19 @@ fires at the early analog read (nf step 2500) or all-done.
   Stage 3: OLMo2 / Dolma recipe -- fully-open reproducible baseline for the paper/collaborators.
   EP scaling knobs carried forward: beta_floor (or qk_norm if it supersedes), possibly double-sided
   nudge at larger scale (cancels O(beta) Taylor bias). $20k/run (Rain) ~ few-B tokens/run.
+
+### RESULT 4 (2026-07-10 06:16): QK-norm validated — parity holds; beta-floor still needed; Stage 1 launched.
+- **Parity with QK-norm (EP-favorable again):** BP+qknorm 1.9253/1.8753/1.9192 = 1.9066;
+  EP+qknorm+beta_floor 1.8588/1.9176/1.8841 = **1.8868 <= BP**. QK-norm preserves EP=BP parity at L12.
+- **ANALOG ANSWER: QK-norm does NOT replace the beta-floor** (they are complementary). EP+qknorm
+  WITHOUT the floor still erodes cos (1.0 -> 0.946 by step 3200) and lands ~0.09 worse CE (2.02 vs
+  1.89). Milder than the old non-QK collapse (0.896) but not fixed. **Why: sig_tok still grows to 21.5
+  even with QK-norm** -- QK-norm normalizes q,k INSIDE attention (bounds the attention LOGITS) but does
+  NOT bound the residual/embedding scale that drives beta_t = beta0*(sig0/sig)^2. So beta_t still
+  collapses -> estimator SNR still needs the floor. QK-norm's payoff is (a) attention logit-bounding
+  (analog softmax device range), (b) scale robustness (logit growth is worse in bigger/deeper models),
+  (c) it is standard OLMo2/Llama -> good for the scale-up. Recipe = **qk_norm + beta_floor together**.
+- **STAGE 1 LAUNCHED (user directive):** stage1_ep_qkbf -- TinyStories full epoch (58,800 steps, 361M
+  tok), qk_norm + beta_floor 3e-4 + cosine, warmup 500, 2.4 it/s solo -> ~6.8 h. The "neng kan"
+  generation demo. Watcher fires at step 10000 (first generation-worthy ckpt) / done / death.
+  Then Stage 2 (FineWeb-Edu) -> Stage 3 (OLMo2).