summaryrefslogtreecommitdiff
path: root/experiments/snapshot_evolution_residual_explosion.py
AgeCommit message (Collapse)Author
2026-04-08Round 34 random-target ablation: Mode 1 fires under random labels tooYurenHao0426
Codex round 34 picked OPTION A (i.i.d. random class targets per minibatch) over the analytic-only OPTION D as the most discriminating test of 'is (a) intrinsic to DFA update geometry or task-driven?'. Smoke test result is unambiguous: ep 0: ||h_L||=8.9 ||g_L||=9.8e-4 ep 1: ||h_L||=1616 ||g_L||=5.1e-6 ep 2: ||h_L||=9768 ||g_L||=8.5e-7 ep 3: ||h_L||=14510 ||g_L||=5.6e-7 (test acc still at chance ~0.07) Three orders of magnitude growth in ||h_L|| in 3 epochs, three orders of magnitude collapse in ||g_L|| in the same 3 epochs, with NO task signal whatsoever — DFA's local-loss geometry is the proximate driver, not data adaptation. - experiments/snapshot_evolution_residual_explosion.py: add --random_targets and --skip_bp flags - paper/main.tex §3 ¶1: replace 'no explicit scale constraint' framing with codex round 34's 6-line geometric argument and the random-target empirical falsifier - paper/main.tex Appendix J: full smoke-test table + interpretation - v2.3: 14 pages total, main content still 8 pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08Round 32+33 H2 ablation: add no_residual_add flag; falsify residual-as-cause ↵YurenHao0426
hypothesis - models/residual_mlp.py: add residual_add and w2_std flags (default unchanged) - experiments/snapshot_evolution_residual_explosion.py: add --no_residual_add and --w2_std CLI flags - paper/main.tex §3 ¶3: add 1-sentence reference to no-residual control showing Mode 1 still fires - paper/main.tex Appendix I: full smoke-test table + interpretation - v2.2 main content stays at 8 pages (within 9-page E&D budget); 13 pages total Smoke test (3 ep, w2_std=0.5, seed 42): - DFA no-residual: ||h_L|| 4.69 -> 22050, ||g|| 1.6e-7 (Mode 1 (a) fires; (b) at floor) - BP no-residual: acc only 0.16 at ep 3 (architecture is partially degenerate) - Conclusion: residual skip is NOT necessary for Mode 1; the proximate trigger is more general - Codex round 33 verdict: WALK BACK H2; demote 100ep run to confirmatory Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>