# Training Time and the Apparent Phase Transition

The first SGD phase-transition figure looked more like a smooth ramp than a
sharp kink. We tested whether this is caused by undertraining.

## Setup

Baseline:

```text
outputs/phase_transition_sgd_256_lr001_T3000
```

Longer probes:

```text
outputs/phase_transition_sgd_long_lr001_T10000_probe
outputs/phase_transition_sgd_verylong_lr001_T30000_probe
```

All use:

```text
task: random-label regression
architecture: 16 -> width -> width -> 4
optimizer: full-batch SGD
learning rate: 0.01
train samples: 128
```

Comparison figures:

```text
outputs/phase_transition_training_time_comparison/train_gap_vs_margin_by_training_time.png
outputs/phase_transition_training_time_comparison/bp_fa_train_loss_by_training_time.png
```

## Observation

At `T=3000`, the gap appears to grow smoothly as hard FA capacity margin
decreases.

At longer training times, the apparent transition shifts left:

| width | FA margin | gap T=3000 | gap T=10000 | gap T=30000 |
|---:|---:|---:|---:|---:|
| 96 | 1026 | 0.009187 | 0.000008 | |
| 64 | 514 | 0.049838 | 0.000470 | |
| 48 | 258 | 0.129947 | 0.005684 | |
| 32 | 2 | 0.297266 | 0.047005 | 0.002913 |
| 24 | -126 | 0.398899 | 0.151777 | 0.044436 |
| 16 | -254 | 0.445812 | 0.407483 | 0.304774 |

So the `T=3000` ramp is partly finite-time optimization. With more training,
positive-margin networks mostly close the FA/BP train gap.

## Interpretation

The hard margin

```text
M_FA = P - K_FA - N*out
```

is a conservative structural regime variable, not a sharp empirical threshold.

The actual transition is shifted to more negative margins because:

1. FA alignment is soft, not a hard rank constraint.
2. Overparameterized networks can absorb some alignment burden through
   redundancy and nonlinear gates.
3. Finite-time optimization can make the transition look smoother and earlier
   than the asymptotic training limit.

Thus:

```text
T=3000: optimization-limited ramp
T=10000/T=30000: sharper and shifted-left capacity transition
```

The current evidence suggests the actual long-time FA train-gap transition lies
between:

```text
M_FA = -126
and
M_FA = -254
```

for this task/architecture, rather than exactly at hard margin zero.

## Consequence for the Paper

Do not present the hard-margin line as an exact transition point.

Better wording:

```text
The hard FA margin gives a conservative capacity-exhaustion boundary. Empirical
train gaps begin to grow as the margin decreases, but the observed transition is
soft and training-time dependent. Longer optimization shifts the effective
transition toward more negative margins, consistent with a soft rather than hard
alignment burden.
```

The phase-transition contribution is still valid, but it should be framed as a
soft transition / regime shift, not a discontinuous kink.

## Next Better Visualization

The most honest visual is the training-time comparison:

```text
train_gap_vs_margin_by_training_time.png
```

It shows both facts:

1. finite-time training produces a smooth ramp;
2. longer training pushes the transition toward the true capacity-limited
   region.

This is stronger than pretending the `T=3000` curve has a sharp kink.