|
User requested: "reproducibility: want an ipynb walkthrough that
reproduces all figures/tables"
reproduce_all.ipynb loads values from saved results/*.json files and
re-derives every cited number and figure in the paper. Cells:
1. Table 1 (5-method audit accuracies, ddof=1)
2. Frozen-blocks baseline (DFA-shallow from resmlp_frozen_blocks_s*.log)
3. §5 matched 30-ep BP/DFA controls + penalty-cost math
4. §4 ¶4 SB/CB/DFA+pen accuracy/cosine/rho
5. §4 ¶4 nudging test 3-seed (from nudging_test_3seed_summary.json)
6. §4 ¶4 training loss decrease 3-seed
7. Appendix M vanilla DFA early-epoch per-layer cosines (layer-0 dominance)
8. §6 ¶1 protocol calibration gaps (24,338× and 63× math)
9. §6 ¶2 fresh-B null calibration
10. §3 ¶3 no-terminal-LN ResMLP control
11-13. Re-render Figure 2 (dissociation), Figure 4 (penalty rescue),
Figure 5 (cross-arch matrix) from their scripts
14. Re-compile main.pdf via tectonic
Every cited number in the paper is traceable to one of the loaded
files, listed in the final summary table. Includes both_stds() helper
that returns (mean, ddof=0, ddof=1) for any list — the paper uses
ddof=1 throughout as of v2.38.
To re-run training, use experiments/ scripts directly; this notebook
is read-only on the saved results.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
