From e65dd2e2460b1da48c83a930304cf9b269fc4447 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Mon, 27 Jul 2026 17:47:13 -0500 Subject: Add AAAI depth experiments and diagnostic figures --- notes/40_reproduction_manifest.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) (limited to 'notes/40_reproduction_manifest.md') diff --git a/notes/40_reproduction_manifest.md b/notes/40_reproduction_manifest.md index 009f058..d7390cb 100644 --- a/notes/40_reproduction_manifest.md +++ b/notes/40_reproduction_manifest.md @@ -54,10 +54,23 @@ deterministic given the seeds shown. Long runs use `nohup`. | kernel code self-test vs autograd (<= 5.3e-15) | `python scripts/real_data_validation.py --self-test` | stdout | 38 | | teacher task: train gap 0.249->0.0069 soft; test gap sign flip (-0.120+/-0.013 at w=32) | `python scripts/downstream_capacity_sweep.py --task mlp --teacher-width 64 --teacher-hidden-layers 2 --normalize-targets --widths 8 12 16 24 32 48 64 96 --train-samples 256 --test-samples 2048 --probe-samples 32 --steps 8000 --lr 0.01 --optimizer sgd --init-seeds 3 --feedback-seeds 8 --data-seed 7 --skip-jacobian --plot --outdir outputs/teacher_test_gap_sgd_T8000` then `python scripts/plot_teacher_test_gap.py` | `teacher_test_gap_sgd_T8000/` | 39 | +## Focused-paper controls added 2026-07-21 + +| claim | command | outputs | note | +|---|---|---|---| +| FA and DFA exact initialization calibration through depth 6: 60 rows, max abs error 0.009164 | `python scripts/aaai_depth_experiments.py --part init --depths 1 2 3 4 6 --width 64 --train-samples 64 --init-seeds 6 --feedback-draws 256 --torch-threads 16 --outdir outputs/aaai_depth_experiments` | `aaai_depth_experiments/initialization_rows.csv` | 43 | +| FA finite-time depth x width grid: matrix-alignment proxy drops 18.06 orders while width-64 gap grows 0.0421 -> 0.3461; frozen MAE 0.06693; two-snapshot MAE 0.01043 / 0.01394 | `python scripts/aaai_depth_experiments.py --part finite --depths 1 2 3 4 6 --finite-widths 32 64 96 --train-samples 64 --finite-init-seeds 4 --finite-feedback-seeds 8 --lr 0.001 --horizon 50 --early-step 20 --torch-threads 16 --outdir outputs/aaai_depth_experiments`, then `--part summarize` | `aaai_depth_experiments/finite_time_*`, `mismatch_vs_cost_by_depth.*` | 43 | +| parameter-matched depth control around P=5376: widths 269/64/47/39/31; gap is nonmonotone and peaks at depth 4 | `python scripts/parameter_matched_depth_control.py --depths 1 2 3 4 6 --reference-depth 2 --reference-width 64 --train-samples 64 --init-seeds 4 --feedback-seeds 8 --lr 0.001 --horizon 50 --early-step 20 --torch-threads 16 --outdir outputs/parameter_matched_depth_control` | `parameter_matched_depth_control/` | 43 | +| initialization-rate compensation reduces but does not remove the gap (29-51% reduction); separate stability-confirmed tuning preserves a gap | `python scripts/learning_rate_compensation.py --depths 1 2 3 4 6 --width 64 --train-samples 128 --steps 200 --base-lr 0.001 --tune-init-seeds 4 --tune-feedback-seeds 4 --eval-init-seeds 6 --eval-feedback-seeds 8 --torch-threads 16 --outdir outputs/learning_rate_compensation` | `learning_rate_compensation/` | 43 | +| cross-entropy CNN initialization identity for FA and DFA: max abs error 0.008309, max standardized error 1.308 | `python scripts/cnn_initialization_validation.py --train-samples 128 --image-size 8 --channels 8 12 --init-seeds 4 --feedback-draws 2048 --torch-threads 8 --outdir outputs/cnn_initialization_validation` | `cnn_initialization_validation/` | 43 | +| two focused three-panel figures | `python scripts/plot_aaai_main_figures.py --outdir outputs/aaai_main_figures` | `aaai_main_figures/` | 43 | + ## Figure -> file map (current best versions) | paper figure (plan in note 41) | file | |---|---| +| focused Figure 1: mismatch vs exact initial cost | `aaai_main_figures/figure1_mismatch_vs_initial_cost.{png,pdf}` | +| focused Figure 2: later cost and operator evolution | `aaai_main_figures/figure2_training_dynamics.{png,pdf}` | | soft ramp + closed form + ramp law | `closed_form_soft_ramp/closed_form_vs_measured_ramp.png` | | e0 moment calibration (synthetic) | `actual_fa_initial_operator_moments/predicted_vs_empirical_initial_erosion_mean.png` | | e0 Gaussian panels | `actual_fa_initial_erosion_distribution/e0_distribution_theory_vs_empirical.png` | -- cgit v1.2.3