Add phase transition trajectory zoom analysis

1 files changed, 122 insertions, 0 deletions

diff --git a/notes/21_more_phase_transition_trajectories.md b/notes/21_more_phase_transition_trajectories.md
new file mode 100644
index 0000000..69bc4ff
--- /dev/null
+++ b/notes/21_more_phase_transition_trajectories.md
@@ -0,0 +1,122 @@
+# More Phase-Transition Trajectories
+
+We ran two larger trajectory sweeps around the capacity transition to separate
+sampling noise from finite-time undertraining.
+
+## Runs
+
+```text
+outputs/phase_transition_zoom_T10000_1024traj
+outputs/phase_transition_zoom_T30000_256traj
+```
+
+The setup is unchanged:
+
+```text
+task: random-label regression
+architecture: 16 -> width -> width -> 4
+optimizer: full-batch SGD
+learning rate: 0.01
+train samples: 128
+```
+
+The `T=10000` run uses:
+
+```text
+widths: 16, 20, 24, 28, 32, 36, 40, 48
+init seeds: 2
+feedback seeds per init: 64
+FA trajectories per width: 128
+total FA trajectories: 1024
+```
+
+The `T=30000` run focuses on the transition:
+
+```text
+widths: 24, 28, 32, 36
+init seeds: 2
+feedback seeds per init: 32
+FA trajectories per width: 64
+total FA trajectories: 256
+```
+
+## Main Plots
+
+```text
+outputs/phase_transition_zoom_comparison/trajectory_gap_clouds_T10000_vs_T30000.png
+outputs/phase_transition_zoom_comparison/trajectory_gap_quantiles_T10000_vs_T30000.png
+```
+
+Per-run plots:
+
+```text
+outputs/phase_transition_zoom_T10000_1024traj/phase_transition_capacity_exhaustion.png
+outputs/phase_transition_zoom_T30000_256traj/phase_transition_capacity_exhaustion.png
+```
+
+## Results
+
+At `T=10000`, the train-gap curve is very smooth and monotone:
+
+| width | FA margin | FA trajectories | train gap mean | train gap std |
+|---:|---:|---:|---:|---:|
+| 16 | -254 | 128 | 0.420780 | 0.075399 |
+| 20 | -190 | 128 | 0.278329 | 0.058114 |
+| 24 | -126 | 128 | 0.171090 | 0.043111 |
+| 28 | -62 | 128 | 0.099732 | 0.033015 |
+| 32 | 2 | 128 | 0.050912 | 0.016008 |
+| 36 | 66 | 128 | 0.030891 | 0.010649 |
+| 40 | 130 | 128 | 0.014464 | 0.004946 |
+| 48 | 258 | 128 | 0.004212 | 0.001836 |
+
+This smooth ramp is not caused by too few feedback seeds. Each point has 128 FA
+trajectories and the standard deviations are much smaller than the mean trend.
+
+At `T=30000`, the near-zero and positive-margin gaps collapse:
+
+| width | FA margin | FA trajectories | train gap mean | train gap std |
+|---:|---:|---:|---:|---:|
+| 24 | -126 | 64 | 0.047899 | 0.026889 |
+| 28 | -62 | 64 | 0.015020 | 0.009012 |
+| 32 | 2 | 64 | 0.003275 | 0.003577 |
+| 36 | 66 | 64 | 0.000932 | 0.001324 |
+
+So the earlier lack of a sharp kink was mostly finite-time undertraining near
+the transition. With longer training, positive-margin networks close the BP/FA
+train gap, while negative-margin networks retain a nonzero gap.
+
+## Interpretation
+
+The hard FA capacity margin is a conservative structural boundary. It is not an
+exact discontinuous transition point.
+
+The empirical picture is:
+
+```text
+T=10000: smooth finite-time ramp
+T=30000: compressed transition; positive margin almost zero, negative margin nonzero
+```
+
+This supports the phase-transition contribution, but the right language is a
+soft capacity transition rather than a hard kink at exactly margin zero.
+
+## Consequence
+
+For the paper figure, the clean visual is the two-panel comparison:
+
+```text
+trajectory_gap_clouds_T10000_vs_T30000.png
+```
+
+It shows:
+
+1. More trajectories do not remove the smooth `T=10000` ramp.
+2. Longer training does remove the apparent positive-margin gap.
+3. The remaining long-time gap is concentrated on the negative-margin side.
+
+This is the strongest current evidence that the original smooth curve mixed two
+effects:
+
+```text
+finite-time optimization gap + structural capacity gap
+```