Add reproduce_all.ipynb: walkthrough for every paper figure + table

2026-04-09T04:05:18+00:00

User requested: "reproducibility: want an ipynb walkthrough that
reproduces all figures/tables"

reproduce_all.ipynb loads values from saved results/*.json files and
re-derives every cited number and figure in the paper. Cells:

  1. Table 1 (5-method audit accuracies, ddof=1)
  2. Frozen-blocks baseline (DFA-shallow from resmlp_frozen_blocks_s*.log)
  3. §5 matched 30-ep BP/DFA controls + penalty-cost math
  4. §4 ¶4 SB/CB/DFA+pen accuracy/cosine/rho
  5. §4 ¶4 nudging test 3-seed (from nudging_test_3seed_summary.json)
  6. §4 ¶4 training loss decrease 3-seed
  7. Appendix M vanilla DFA early-epoch per-layer cosines (layer-0 dominance)
  8. §6 ¶1 protocol calibration gaps (24,338× and 63× math)
  9. §6 ¶2 fresh-B null calibration
  10. §3 ¶3 no-terminal-LN ResMLP control
  11-13. Re-render Figure 2 (dissociation), Figure 4 (penalty rescue),
         Figure 5 (cross-arch matrix) from their scripts
  14. Re-compile main.pdf via tectonic

Every cited number in the paper is traceable to one of the loaded
files, listed in the final summary table. Includes both_stds() helper
that returns (mean, ddof=0, ddof=1) for any list — the paper uses
ddof=1 throughout as of v2.38.

To re-run training, use experiments/ scripts directly; this notebook
is read-only on the saved results.

Co-Authored-By: Claude Opus 4.6 (1M context)

faeval.git/reproduce_all.ipynb, branch master

Add reproduce_all.ipynb: walkthrough for every paper figure + table