summaryrefslogtreecommitdiff
path: root/notes/40_reproduction_manifest.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-06-09 21:35:51 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-06-09 21:35:51 -0500
commitf860ba2ffb409dc1b7dfab9534ed05cb25692269 (patch)
treed30029bb8bb754a7c3c5f44fd13869e4e4178b84 /notes/40_reproduction_manifest.md
parent84fa70be406248e660488ddc4c56ca5ea0034189 (diff)
Organize repo and add AAAI-27 writing plan
Root README orients the repo; notes/README reindexed by status (writing pipeline / authoritative note per result / six historical phases with superseded markers); scripts/README gains a status table. Note 40 = reproduction manifest (claim -> command -> outputs -> note; everything unlisted is not citable). Note 41 = the binding paper plan: AAAI-27 constraints (7pp + refs, appendix not guaranteed read), narrative, section/page budget, frozen theorem and figure numbering, related-work list with per-cite rationale, writing rules, weekly schedule to the 07-20/07-27 deadlines, and risk register. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Diffstat (limited to 'notes/40_reproduction_manifest.md')
-rw-r--r--notes/40_reproduction_manifest.md83
1 files changed, 83 insertions, 0 deletions
diff --git a/notes/40_reproduction_manifest.md b/notes/40_reproduction_manifest.md
new file mode 100644
index 0000000..7e18d98
--- /dev/null
+++ b/notes/40_reproduction_manifest.md
@@ -0,0 +1,83 @@
+# Reproduction Manifest (paper-grade runs only)
+
+One row per number/figure that can appear in the paper: the claim, the exact
+command (or the note that records it), the output directory, and the note that
+interprets it. `outputs/` is gitignored — these runs are the regenerable
+ground truth behind the ledger (note 36). Anything in `outputs/` NOT listed
+here is an exploratory/superseded/smoke run and must not be cited.
+
+Environment: python 3.13, torch (CPU, float64), `requirements.txt`. All runs
+deterministic given the seeds shown. Long runs use `nohup`.
+
+## Static geometry (T1-T3 + overlap)
+
+| claim | command | outputs | note |
+|---|---|---|---|
+| Beta law, tails, multilayer product (700k samples, D=64..4096) | `python scripts/capacity_empirical_validation.py --dimensions 64 128 256 512 1024 2048 4096 --samples 100000 --batch-size 2048 --seed 123 --plot` | `capacity_empirical_validation/` | 02 |
+| surprisal `S~Exp(1)`, `sum S ~ Gamma(L,1)` (max KS 0.0042) | `python scripts/multilayer_capacity_distribution.py --dimensions 64 256 1024 4096 --layers 1 2 4 8 16 --samples 100000 --batch-size 8192 --seed 456 --plot` | `multilayer_capacity_distribution/` | 02 |
+| minimax: isotropic attains 1/D, alternatives below | `python scripts/minimax_initialization.py --dimension 32 --feedback-samples 20000 --target-samples 10000 --seed 11 --subspace-dim 4 --plot` | `minimax_initialization/` | 02 |
+| coverage distributions per scheme | `python scripts/initialization_distribution_matching.py --dimension 128 --target-samples 100000 --feedback-samples 100000 --batch-size 8192 --seed 2026 --subspace-dim 8 --anisotropy 64 --plot` | `initialization_distribution_matching/` | 02 |
+| hard rank loss + kd/P overlap | `python scripts/functional_capacity_overlap.py --parameters 96 --task-rank 24 --constraint-ranks 0 24 48 72 84 96 --trials 100 --seed 5 --plot` | `functional_capacity_overlap/` | 02 |
+
+## The soft ramp (dense sweep) + closed form (T7)
+
+| claim | command | outputs | note |
+|---|---|---|---|
+| dense T=30000 ramp: gap 0.1468 -> 0.000162, smooth, log-linear R^2=0.993 | `python scripts/downstream_capacity_sweep.py --task random --widths 20 22 24 26 28 30 32 34 36 38 40 --train-samples 128 --test-samples 1024 --probe-samples 64 --steps 30000 --lr 0.01 --optimizer sgd --init-seeds 1 --feedback-seeds 32 --data-seed 0 --skip-jacobian --plot --outdir outputs/phase_transition_dense_T30000_352traj` (config preserved in its `summary.json`) | `phase_transition_dense_T30000_352traj/` | 22 |
+| closed form, matched data+init: corr 0.933 matched / 0.982 5-init geo mean; lam_min ensemble corr 0.987 | `python scripts/closed_form_soft_ramp.py --torch-threads 8` (reads the dense dir) | `closed_form_soft_ramp/` | 34, 37 |
+| hard-k null model overpredicts (2x-250x) | `python scripts/soft_erosion_theory_vs_empirical.py` (reads the dense dir; see note 25 for the recorded run) | `soft_erosion_theory_vs_empirical/` | 25 |
+
+## Initial erosion (T5 + T6)
+
+| claim | command | outputs | note |
+|---|---|---|---|
+| `E_B[e_0]=0.4378+/-0.008` vs hidden share 0.4403; per-width errors <= 0.008 | `python scripts/actual_fa_initial_operator_moments.py --widths 16 24 32 48 64 96 --init-seeds 6 --feedback-samples 512 --torch-threads 8 --outdir outputs/actual_fa_initial_operator_moments` | `actual_fa_initial_operator_moments/` | 29 |
+| e0 exactly Gaussian; KS p 0.13-0.999; sampler = real FA backward, linear identity <= 8.9e-16 | `python scripts/actual_fa_initial_erosion_distribution.py --widths 16 32 64 128 --init-seeds 3 --feedback-samples 4096 --torch-threads 8 --outdir outputs/actual_fa_initial_erosion_distribution` | `actual_fa_initial_erosion_distribution/` (pre-fix snapshot kept in `*_pre_fix_backup/`) | 32, 37 |
+
+## Finite-time chain (lazy validity -> drift -> estimator)
+
+| claim | command | outputs | note |
+|---|---|---|---|
+| T=5 frozen-K(0) local exactness: corr 0.99947, MAE 0.00083; horizon table T=1..50 | recorded invocation in note 10 (fixed width 64, N in {64..320}, SGD lr 1e-3, 4 init x 8 feedback) | `downstream_capacity_fixed_width64_sgd_T5_N*/`, `fa_tangent_kernel_fixed_width64_sgd_T5_*` | 10 |
+| oracle time-varying K(t) removes T=50 error: gap MAE 0.0277 -> 0.000273 | `python scripts/finite_time_kernel_diagnostic.py --width 64 --train-samples 128 --steps 50 --lr 1e-3 --init-seeds 2 --feedback-seeds 4` | `finite_time_kernel_probe_T50_N128_8runs/` | 13 |
+| scalar early predictors fail across N (corr 0.124) | `python scripts/early_kernel_predictors.py ...` (recorded in note 14) | `early_directional_predictors_T50_width64_N*/` | 14 |
+| estimator, 256 traj: velocity MAE 0.00189, corr 0.99885; fixed K(0) 10x worse; retangent upper bracket | `python scripts/compressed_operator_predictor.py --width 64 --hidden-layers 2 --train-samples <N> --target-steps 50 --early-steps 20 --lr 1e-3 --init-seeds 4 --feedback-seeds 8` for N in {64,96,128,160,192,224,256,320}, then `python scripts/plot_compressed_operator_results.py` | `compressed_operator_s20_256traj_T50_width64_N*/`, `*_plots/` | 15 |
+| stress grid: corr 0.994-0.99989 across depth {1,2,3} x width {32,64,96} x horizon {25,50,100} | per-setting runs recorded in note 16, then `python scripts/summarize_operator_stress_grid.py` + `python scripts/plot_operator_overlap_grid.py` | `stress_depth*_*/`, `operator_stress_grid_summary/` | 16, 17 |
+| first-order derivative extrapolation diverges (MAE 270) | `python scripts/fa_tangent_hierarchy_derivative_probe.py` (eps 0.05; recorded in note 31) | `fa_tangent_hierarchy_derivative_probe_eps005/` | 31 |
+
+## Real data + generalization
+
+| claim | command | outputs | note |
+|---|---|---|---|
+| MNIST e0: 16 configs, share 0.39-0.82, abs diff <= 0.0089 | `python scripts/real_data_validation.py --part a --torch-threads 16` | `real_data_validation_mnist/e0_mnist_*` | 38 |
+| MNIST estimator: 96 traj, velocity MAE 0.00196 (3.9% rel), bias -0.0006, corr 0.911; fixed MAE 0.0061 | `python scripts/real_data_validation.py --part b --est-widths 128 256 --lr-safety 0.05 --torch-threads 16` | `real_data_validation_mnist/estimator_mnist_*` | 38 |
+| kernel code self-test vs autograd (<= 5.3e-15) | `python scripts/real_data_validation.py --self-test` | stdout | 38 |
+| teacher task: train gap 0.249->0.0069 soft; test gap sign flip (-0.120+/-0.013 at w=32) | `python scripts/downstream_capacity_sweep.py --task mlp --teacher-width 64 --teacher-hidden-layers 2 --normalize-targets --widths 8 12 16 24 32 48 64 96 --train-samples 256 --test-samples 2048 --probe-samples 32 --steps 8000 --lr 0.01 --optimizer sgd --init-seeds 3 --feedback-seeds 8 --data-seed 7 --skip-jacobian --plot --outdir outputs/teacher_test_gap_sgd_T8000` then `python scripts/plot_teacher_test_gap.py` | `teacher_test_gap_sgd_T8000/` | 39 |
+
+## Figure -> file map (current best versions)
+
+| paper figure (plan in note 41) | file |
+|---|---|
+| soft ramp + closed form + ramp law | `closed_form_soft_ramp/closed_form_vs_measured_ramp.png` |
+| e0 moment calibration (synthetic) | `actual_fa_initial_operator_moments/predicted_vs_empirical_initial_erosion_mean.png` |
+| e0 Gaussian panels | `actual_fa_initial_erosion_distribution/e0_distribution_theory_vs_empirical.png` |
+| estimator scatter (synthetic 256 traj) | `compressed_operator_s20_256traj_T50_width64_plots/T50_s20_prediction_scatter.png` |
+| estimator trajectories + band | `compressed_operator_s20_256traj_T50_width64_plots/T50_s20_velocity_theory_band_empirical_trajectories.png` |
+| MNIST e0 calibration | `real_data_validation_mnist/e0_mnist_calibration.png` |
+| MNIST estimator scatter | `real_data_validation_mnist/estimator_mnist_scatter.png` |
+| teacher train vs test gap | `teacher_test_gap_sgd_T8000/teacher_train_vs_test_gap.png` |
+| static beta (appendix) | `capacity_empirical_validation/` plots |
+| stress grid (appendix) | `operator_stress_grid_summary/stress_grid_*` |
+
+All paper figures will be re-rendered to AAAI column dimensions during the
+writing pass (note 41, week 2); the files above are the data-correct sources.
+
+## Not paper-grade (do not cite)
+
+Everything else under `outputs/`, in particular: all `*smoke*`, all
+`downstream_capacity_random_*` / `downstream_capacity_sweep*` Adam-era sweeps
+(superseded by SGD runs; notes 05-07), `phase_transition_sgd_*` short-T probes
+(superseded by the dense run), `compressed_operator_T50_*` and
+`*_retangent_*` intermediate variants (subsumed by the s20 256-traj set +
+stress grid), `trajectory_*` bridge-era runs (notes 02/08; historical), and
+`fa_tangent_hierarchy_first_order_try/` (negative result recorded in note 31).