diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-15 21:27:03 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-15 21:27:03 -0500 |
| commit | 06e3f1f435fe59e1e66d01cf937051f75e49f118 (patch) | |
| tree | 5d28ebb09a73e2e9df9d34e8a848665044cb96f9 /docs/campaign/CASCADE_ABLATION_PLAN.md | |
| parent | 90dde02cf7a3c043940f9f6c4bfa99f659e937ea (diff) | |
RESULT 27: K-saturation across training stages — K3=K8 at s5k/50k/100k; fixed-point bias (not truncation) grows; answers Alexi's more-nudges challenge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs/campaign/CASCADE_ABLATION_PLAN.md')
| -rw-r--r-- | docs/campaign/CASCADE_ABLATION_PLAN.md | 19 |
1 files changed, 19 insertions, 0 deletions
diff --git a/docs/campaign/CASCADE_ABLATION_PLAN.md b/docs/campaign/CASCADE_ABLATION_PLAN.md index 215d586..d8e5bb3 100644 --- a/docs/campaign/CASCADE_ABLATION_PLAN.md +++ b/docs/campaign/CASCADE_ABLATION_PLAN.md @@ -568,6 +568,25 @@ direction), not training-under-fault; wave-2 = co-training with faults injected +### RESULT 27 (2026-07-15): K-SATURATION ACROSS TRAINING (Alexi's challenge answered with data). +Challenge (Alexi Gladstone): K=3 nudge sweeps seems very few; as training roughens the landscape, +more sweeps may be needed absent a convexity argument. +Probe: probe_blockcos.py on fw72m_cent ckpts s5000/s50000/s100000, K in {1,2,3,8}, beta 3e-3, 4 batches. +| ckpt | K=1 | K=2 | K=3 | K=8 | +|---|---|---|---|---| +| s5000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | +| s50000 | 1.0000 | 0.9998 | 0.9997 | 0.9997 | +| s100000 | 1.0000 | 0.9994 | 0.9995 | 0.9995 | +Verdicts: (a) fixed point reached by K~2-3 at EVERY stage; K3=K8 to 4 decimals — no K-starvation +trend over 100k steps. (b) What grows with training is the FIXED POINT's own O(beta) transmission +bias (1.0000 -> 0.9995) — the window story, not truncation; K cannot treat it, beta/centered can. +(c) ANCHOR stays exactly 1.0000 at all stages: RESULT 25 decomposition holds at 72M throughout. +(d) K=1 constant 1.0000 = the frozen-state BP degeneracy (digital shortcut, not physics). +(e) Cost note: K=2 is already ~at the fixed point -> potential ~25% nudged-phase saving +(validation item, recipe unchanged for now). The correct response to a roughening landscape is +governed beta (the certificate is the live rho-hat spectral meter), not more sweeps — at the +edge, extra sweeps amplify (w2_adapt). + ### ERRATUM to RESULT 21 (found 2026-07-15): the original fw72m's best val 3.7117 was first hit at step 104,800 (log line 1055), NOT at s185000 (that was the checkpoint used for sampling). Best-val progression: 3.8769@30.9k -> 3.8328@61.3k -> 3.8004@75.3k -> 3.7789@83.3k -> |