diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 20 |
1 files changed, 20 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 52c266e..0df3e72 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -568,6 +568,26 @@ direction), not training-under-fault; wave-2 = co-training with faults injected +### RESULT 69 (2026-07-29): SATURATION-BLINDNESS REFUTED BY ITS OWN TREATMENT TEST; the +leak survives exclusion #11; dgain (displacement-only amplification) is the live arm. +Battery 4 (beta dose): leak INVERSE in beta — 1.5e-3: 0.0615, 3e-3: 0.0462, 6e-3: 0.0352 +(~1/beta additive signature). This motivated the saturation-blindness hypothesis (sharp +softmax switches invisible to finite displacement; the sibling Hopfield-EP "saturated units +EP!=BPTT 80-130deg" lesson at transformer scale). +Battery 5 (user-approved, matched-cap BP bar): logit softcap 30/50 (Gemma-2 style; firing +certificate: cap=1 destroys the model, val 6.45) closes 0.0000 of the leak — relative leak +0.0462 IDENTICAL to 4 decimals. Saturation-at-the-logit level is NOT the mechanism. BONUS: +the cap's own price under BP ~ 0 in-window (allbp+cap30 3.5205 vs allbp 3.5206) — the user's +"cap hurts performance" concern priced at ~zero here (Gemma prior confirmed). +SURVIVING CONSTRAINTS: top-half (100%), attn-dominant (2/3), ~1/beta, immune to: logit cap, +odd/even bias, read noise, K, precision, head surgery, bottom blocks, magnitude/clip, Muon, +direction (FA argument). Threshold nonlinearity still indicated by 1/beta but NOT at softmax +logits — candidates: ffn gates (mixffn arm measuring its share), norms, qk-norm. +Battery 6 IN FLIGHT: leak_dg4top (top-half state-formation d x4, cotangents true-d; smoke +shows the price: gate cos 0.83 from anchor second-order — CE judges), leak_dg2all +(displacement-vs-force discriminator vs the b6d arm which doubled BOTH: b6d 3.5558), +leak_mixffn (completes the R44-style partition additivity). + ### RESULT 68 (2026-07-28): LEAK LOCALIZED — 100% in the TOP-HALF block gradients, ~2/3 attention; the SAME map as R44 at 72M, 6x stronger at width. One structure, two scales. Battery 3 (guaranteed-signal partition, bars allbp 3.5206 / EP 3.5668): mixtop closes 100% |