From 7467b5c313393e9c01b5d877f45e680e0c35213f Mon Sep 17 00:00:00 2001 From: Yuren Hao Date: Wed, 29 Jul 2026 04:46:31 -0500 Subject: =?UTF-8?q?RESULT=2069:=20=E9=A5=B1=E5=92=8C=E7=9B=B2=E5=8C=BA?= =?UTF-8?q?=E8=A2=AB=E8=87=AA=E5=B7=B1=E7=9A=84=E6=B2=BB=E7=96=97=E6=B5=8B?= =?UTF-8?q?=E8=AF=95=E8=AF=81=E4=BC=AA(cap30/50=E5=85=B3=E9=97=AD0.0000,?= =?UTF-8?q?=E8=AF=81=E4=B9=A6cap=3D1=E7=88=86=E6=A8=A1=E5=9E=8B);=20cap?= =?UTF-8?q?=E4=BB=B7=E6=A0=BC=E2=89=880(=E7=94=A8=E6=88=B7=E6=8B=85?= =?UTF-8?q?=E5=BF=A7=E5=AE=9A=E4=BB=B7);=20=E5=AD=98=E6=B4=BB=E7=BA=A6?= =?UTF-8?q?=E6=9D=9F=3D=E9=A1=B6=E5=8D=8A+attn+1/=CE=B2+11=E9=A1=B9?= =?UTF-8?q?=E5=85=8D=E7=96=AB;=20dgain=E4=BD=8D=E7=A7=BB=E8=87=82=E5=9C=A8?= =?UTF-8?q?=E9=A3=9E(cos0.83=E7=9A=84=E4=BB=A3=E4=BB=B7=E7=94=B1CE?= =?UTF-8?q?=E8=A3=81=E5=86=B3)+mixffn=E8=A1=A5=E5=88=86=E8=A7=A3?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn --- docs/campaign/CASCADE_ABLATION_PLAN.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) (limited to 'docs/campaign/CASCADE_ABLATION_PLAN.md') diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 52c266e..0df3e72 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -568,6 +568,26 @@ direction), not training-under-fault; wave-2 = co-training with faults injected +### RESULT 69 (2026-07-29): SATURATION-BLINDNESS REFUTED BY ITS OWN TREATMENT TEST; the +leak survives exclusion #11; dgain (displacement-only amplification) is the live arm. +Battery 4 (beta dose): leak INVERSE in beta — 1.5e-3: 0.0615, 3e-3: 0.0462, 6e-3: 0.0352 +(~1/beta additive signature). This motivated the saturation-blindness hypothesis (sharp +softmax switches invisible to finite displacement; the sibling Hopfield-EP "saturated units +EP!=BPTT 80-130deg" lesson at transformer scale). +Battery 5 (user-approved, matched-cap BP bar): logit softcap 30/50 (Gemma-2 style; firing +certificate: cap=1 destroys the model, val 6.45) closes 0.0000 of the leak — relative leak +0.0462 IDENTICAL to 4 decimals. Saturation-at-the-logit level is NOT the mechanism. BONUS: +the cap's own price under BP ~ 0 in-window (allbp+cap30 3.5205 vs allbp 3.5206) — the user's +"cap hurts performance" concern priced at ~zero here (Gemma prior confirmed). +SURVIVING CONSTRAINTS: top-half (100%), attn-dominant (2/3), ~1/beta, immune to: logit cap, +odd/even bias, read noise, K, precision, head surgery, bottom blocks, magnitude/clip, Muon, +direction (FA argument). Threshold nonlinearity still indicated by 1/beta but NOT at softmax +logits — candidates: ffn gates (mixffn arm measuring its share), norms, qk-norm. +Battery 6 IN FLIGHT: leak_dg4top (top-half state-formation d x4, cotangents true-d; smoke +shows the price: gate cos 0.83 from anchor second-order — CE judges), leak_dg2all +(displacement-vs-force discriminator vs the b6d arm which doubled BOTH: b6d 3.5558), +leak_mixffn (completes the R44-style partition additivity). + ### RESULT 68 (2026-07-28): LEAK LOCALIZED — 100% in the TOP-HALF block gradients, ~2/3 attention; the SAME map as R44 at 72M, 6x stronger at width. One structure, two scales. Battery 3 (guaranteed-signal partition, bars allbp 3.5206 / EP 3.5668): mixtop closes 100% -- cgit v1.2.3