Add phase transition trajectory zoom analysis

2 files changed, 265 insertions, 0 deletions

diff --git a/notes/21_more_phase_transition_trajectories.md b/notes/21_more_phase_transition_trajectories.md
new file mode 100644
index 0000000..69bc4ff
--- /dev/null
+++ b/notes/21_more_phase_transition_trajectories.md
@@ -0,0 +1,122 @@
+# More Phase-Transition Trajectories
+
+We ran two larger trajectory sweeps around the capacity transition to separate
+sampling noise from finite-time undertraining.
+
+## Runs
+
+```text
+outputs/phase_transition_zoom_T10000_1024traj
+outputs/phase_transition_zoom_T30000_256traj
+```
+
+The setup is unchanged:
+
+```text
+task: random-label regression
+architecture: 16 -> width -> width -> 4
+optimizer: full-batch SGD
+learning rate: 0.01
+train samples: 128
+```
+
+The `T=10000` run uses:
+
+```text
+widths: 16, 20, 24, 28, 32, 36, 40, 48
+init seeds: 2
+feedback seeds per init: 64
+FA trajectories per width: 128
+total FA trajectories: 1024
+```
+
+The `T=30000` run focuses on the transition:
+
+```text
+widths: 24, 28, 32, 36
+init seeds: 2
+feedback seeds per init: 32
+FA trajectories per width: 64
+total FA trajectories: 256
+```
+
+## Main Plots
+
+```text
+outputs/phase_transition_zoom_comparison/trajectory_gap_clouds_T10000_vs_T30000.png
+outputs/phase_transition_zoom_comparison/trajectory_gap_quantiles_T10000_vs_T30000.png
+```
+
+Per-run plots:
+
+```text
+outputs/phase_transition_zoom_T10000_1024traj/phase_transition_capacity_exhaustion.png
+outputs/phase_transition_zoom_T30000_256traj/phase_transition_capacity_exhaustion.png
+```
+
+## Results
+
+At `T=10000`, the train-gap curve is very smooth and monotone:
+
+| width | FA margin | FA trajectories | train gap mean | train gap std |
+|---:|---:|---:|---:|---:|
+| 16 | -254 | 128 | 0.420780 | 0.075399 |
+| 20 | -190 | 128 | 0.278329 | 0.058114 |
+| 24 | -126 | 128 | 0.171090 | 0.043111 |
+| 28 | -62 | 128 | 0.099732 | 0.033015 |
+| 32 | 2 | 128 | 0.050912 | 0.016008 |
+| 36 | 66 | 128 | 0.030891 | 0.010649 |
+| 40 | 130 | 128 | 0.014464 | 0.004946 |
+| 48 | 258 | 128 | 0.004212 | 0.001836 |
+
+This smooth ramp is not caused by too few feedback seeds. Each point has 128 FA
+trajectories and the standard deviations are much smaller than the mean trend.
+
+At `T=30000`, the near-zero and positive-margin gaps collapse:
+
+| width | FA margin | FA trajectories | train gap mean | train gap std |
+|---:|---:|---:|---:|---:|
+| 24 | -126 | 64 | 0.047899 | 0.026889 |
+| 28 | -62 | 64 | 0.015020 | 0.009012 |
+| 32 | 2 | 64 | 0.003275 | 0.003577 |
+| 36 | 66 | 64 | 0.000932 | 0.001324 |
+
+So the earlier lack of a sharp kink was mostly finite-time undertraining near
+the transition. With longer training, positive-margin networks close the BP/FA
+train gap, while negative-margin networks retain a nonzero gap.
+
+## Interpretation
+
+The hard FA capacity margin is a conservative structural boundary. It is not an
+exact discontinuous transition point.
+
+The empirical picture is:
+
+```text
+T=10000: smooth finite-time ramp
+T=30000: compressed transition; positive margin almost zero, negative margin nonzero
+```
+
+This supports the phase-transition contribution, but the right language is a
+soft capacity transition rather than a hard kink at exactly margin zero.
+
+## Consequence
+
+For the paper figure, the clean visual is the two-panel comparison:
+
+```text
+trajectory_gap_clouds_T10000_vs_T30000.png
+```
+
+It shows:
+
+1. More trajectories do not remove the smooth `T=10000` ramp.
+2. Longer training does remove the apparent positive-margin gap.
+3. The remaining long-time gap is concentrated on the negative-margin side.
+
+This is the strongest current evidence that the original smooth curve mixed two
+effects:
+
+```text
+finite-time optimization gap + structural capacity gap
+```
diff --git a/scripts/plot_phase_transition_zoom_comparison.py b/scripts/plot_phase_transition_zoom_comparison.py
new file mode 100644
index 0000000..8e0dc03
--- /dev/null
+++ b/scripts/plot_phase_transition_zoom_comparison.py
@@ -0,0 +1,143 @@
+#!/usr/bin/env python3
+"""Plot trajectory-level capacity transition at two training horizons."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import matplotlib.pyplot as plt
+import numpy as np
+import pandas as pd
+
+
+RUNS = [
+    (
+        "T=10000, 1024 FA trajectories",
+        Path("outputs/phase_transition_zoom_T10000_1024traj"),
+        "#c65f16",
+    ),
+    (
+        "T=30000, 256 FA trajectories",
+        Path("outputs/phase_transition_zoom_T30000_256traj"),
+        "#174f91",
+    ),
+]
+
+
+def summarize_gaps(runs: pd.DataFrame) -> pd.DataFrame:
+    fa = runs[runs["run_type"] == "fa"].copy()
+    records = []
+    for margin, group in fa.groupby("fa_capacity_margin"):
+        gaps = group["train_gap_to_bp"].to_numpy(dtype=np.float64)
+        records.append(
+            {
+                "margin": float(margin),
+                "mean": float(gaps.mean()),
+                "q05": float(np.quantile(gaps, 0.05)),
+                "q25": float(np.quantile(gaps, 0.25)),
+                "q75": float(np.quantile(gaps, 0.75)),
+                "q95": float(np.quantile(gaps, 0.95)),
+            }
+        )
+    return pd.DataFrame(records).sort_values("margin", ascending=False)
+
+
+def add_summary(ax: plt.Axes, runs: pd.DataFrame, label: str, color: str) -> pd.DataFrame:
+    summary = summarize_gaps(runs)
+    x = summary["margin"].to_numpy(dtype=np.float64)
+    ax.fill_between(
+        x,
+        summary["q05"],
+        summary["q95"],
+        color=color,
+        alpha=0.13,
+        linewidth=0,
+    )
+    ax.fill_between(
+        x,
+        summary["q25"],
+        summary["q75"],
+        color=color,
+        alpha=0.24,
+        linewidth=0,
+    )
+    ax.plot(
+        x,
+        summary["mean"],
+        color=color,
+        marker="o",
+        linewidth=2.3,
+        label=label,
+    )
+    return summary
+
+
+def add_trajectories(ax: plt.Axes, runs: pd.DataFrame, color: str) -> None:
+    fa = runs[runs["run_type"] == "fa"].copy()
+    fa["_tid"] = fa["init_seed"].astype(str) + ":" + fa["feedback_seed"].astype(str)
+    for _tid, group in fa.groupby("_tid"):
+        group = group.sort_values("fa_capacity_margin", ascending=False)
+        ax.plot(
+            group["fa_capacity_margin"],
+            group["train_gap_to_bp"],
+            color=color,
+            alpha=0.11,
+            linewidth=0.85,
+        )
+
+
+def main() -> None:
+    outdir = Path("outputs/phase_transition_zoom_comparison")
+    outdir.mkdir(parents=True, exist_ok=True)
+
+    fig, axes = plt.subplots(1, 2, figsize=(13.2, 5.2), dpi=180)
+    for ax, (label, run_dir, color) in zip(axes, RUNS, strict=True):
+        runs = pd.read_csv(run_dir / "runs.csv")
+        add_trajectories(ax, runs, color)
+        summary = add_summary(ax, runs, label, color)
+        ax.axvline(0, color="black", linestyle="--", linewidth=1.1)
+        ax.axhline(0, color="black", linewidth=1.0)
+        ax.set_xlabel("hard FA capacity margin P - K_FA - N*out")
+        ax.set_title(label)
+        ax.grid(alpha=0.18)
+        ax.legend(frameon=True)
+        fa = runs[runs["run_type"] == "fa"]
+        ymax = max(
+            0.01,
+            float(summary["q95"].max()) * 1.18,
+            float(fa["train_gap_to_bp"].max()) * 1.03,
+        )
+        ax.set_ylim(-0.004 * ymax, ymax)
+    axes[0].set_xlim(285, -280)
+    axes[1].set_xlim(90, -150)
+    for ax in axes:
+        ax.set_ylabel("FA train MSE - BP train MSE")
+    fig.suptitle("More trajectories separate finite-time ramp from capacity-limited gap")
+    fig.tight_layout()
+    path = outdir / "trajectory_gap_clouds_T10000_vs_T30000.png"
+    fig.savefig(path)
+    plt.close(fig)
+
+    fig2, ax = plt.subplots(figsize=(8.8, 5.3), dpi=180)
+    for label, run_dir, color in RUNS:
+        runs = pd.read_csv(run_dir / "runs.csv")
+        add_summary(ax, runs, label, color)
+    ax.axvline(0, color="black", linestyle="--", linewidth=1.1, label="hard margin=0")
+    ax.axhline(0, color="black", linewidth=1.0)
+    ax.set_xlim(285, -280)
+    ax.set_xlabel("hard FA capacity margin P - K_FA - N*out")
+    ax.set_ylabel("FA train MSE - BP train MSE")
+    ax.set_title("Training horizon compresses the apparent transition")
+    ax.grid(alpha=0.18)
+    ax.legend(frameon=True)
+    fig2.tight_layout()
+    path2 = outdir / "trajectory_gap_quantiles_T10000_vs_T30000.png"
+    fig2.savefig(path2)
+    plt.close(fig2)
+
+    print(f"plot: {path}")
+    print(f"plot: {path2}")
+
+
+if __name__ == "__main__":
+    main()