diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-29 04:46:31 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-29 04:46:31 -0500 |
| commit | 7467b5c313393e9c01b5d877f45e680e0c35213f (patch) | |
| tree | 8aa466c1b308170361c9ce353439dbc191dab2ff /docs | |
| parent | c67f91ba8f1eb70c64897641bb88cee4af249ed2 (diff) | |
RESULT 69: 饱和盲区被自己的治疗测试证伪(cap30/50关闭0.0000,证书cap=1爆模型); cap价格≈0(用户担忧定价); 存活约束=顶半+attn+1/β+11项免疫; dgain位移臂在飞(cos0.83的代价由CE裁决)+mixffn补分解
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 20 |
1 files changed, 20 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 52c266e..0df3e72 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -568,6 +568,26 @@ direction), not training-under-fault; wave-2 = co-training with faults injected +### RESULT 69 (2026-07-29): SATURATION-BLINDNESS REFUTED BY ITS OWN TREATMENT TEST; the +leak survives exclusion #11; dgain (displacement-only amplification) is the live arm. +Battery 4 (beta dose): leak INVERSE in beta — 1.5e-3: 0.0615, 3e-3: 0.0462, 6e-3: 0.0352 +(~1/beta additive signature). This motivated the saturation-blindness hypothesis (sharp +softmax switches invisible to finite displacement; the sibling Hopfield-EP "saturated units +EP!=BPTT 80-130deg" lesson at transformer scale). +Battery 5 (user-approved, matched-cap BP bar): logit softcap 30/50 (Gemma-2 style; firing +certificate: cap=1 destroys the model, val 6.45) closes 0.0000 of the leak — relative leak +0.0462 IDENTICAL to 4 decimals. Saturation-at-the-logit level is NOT the mechanism. BONUS: +the cap's own price under BP ~ 0 in-window (allbp+cap30 3.5205 vs allbp 3.5206) — the user's +"cap hurts performance" concern priced at ~zero here (Gemma prior confirmed). +SURVIVING CONSTRAINTS: top-half (100%), attn-dominant (2/3), ~1/beta, immune to: logit cap, +odd/even bias, read noise, K, precision, head surgery, bottom blocks, magnitude/clip, Muon, +direction (FA argument). Threshold nonlinearity still indicated by 1/beta but NOT at softmax +logits — candidates: ffn gates (mixffn arm measuring its share), norms, qk-norm. +Battery 6 IN FLIGHT: leak_dg4top (top-half state-formation d x4, cotangents true-d; smoke +shows the price: gate cos 0.83 from anchor second-order — CE judges), leak_dg2all +(displacement-vs-force discriminator vs the b6d arm which doubled BOTH: b6d 3.5558), +leak_mixffn (completes the R44-style partition additivity). + ### RESULT 68 (2026-07-28): LEAK LOCALIZED — 100% in the TOP-HALF block gradients, ~2/3 attention; the SAME map as R44 at 72M, 6x stronger at width. One structure, two scales. Battery 3 (guaranteed-signal partition, bars allbp 3.5206 / EP 3.5668): mixtop closes 100% |