# Fixed-Width N-Sweep Probe

The previous width-sweep short-SGD plot showed that tangent-kernel predictions
were systematically above empirical FA/BP train gaps. A decomposition showed
the main error source was not FA:

\[
\mathbb E[L_{\mathrm{FA}}^{\mathrm{pred}}-L_{\mathrm{FA}}^{\mathrm{emp}}]
\approx
0.008,
\]

while fixed-kernel BP was too optimistic:

\[
\mathbb E[L_{\mathrm{BP}}^{\mathrm{pred}}-L_{\mathrm{BP}}^{\mathrm{emp}}]
\approx
-0.048.
\]

So the predicted gap was high mainly because the BP baseline was predicted too
low.

## Revised Design

Do not shrink capacity by shrinking width first. That changes two things at
once:

1. the FA capacity margin;
2. the finite-width quality of the BP tangent-kernel approximation.

Instead, keep the MLP width fixed and vary the number of training samples:

\[
P-K_{\mathrm{FA}}-Nn_L.
\]

This changes the capacity margin while keeping finite-width BP bias roughly
controlled.

Probe settings:

- width \(n=64\);
- random-label regression;
- \(N\in\{64,128,192,256,320\}\);
- output dimension \(4\);
- SGD, learning rate \(10^{-3}\), steps \(50\);
- 2 init seeds and 4 feedback seeds per init.

## Result

Combined output:

`outputs/fa_tangent_kernel_fixed_width64_sgd_short_combined/fixed_width64_N_sweep_theory_empirical_overlay.png`

The fixed-width probe improves the calibration:

\[
\operatorname{corr}
\left(
\Delta L^{\mathrm{pred}},
\Delta L^{\mathrm{emp}}
\right)
=
0.983.
\]

\[
\operatorname{MAE}=0.0244.
\]

The linear fit becomes

\[
\Delta L^{\mathrm{emp}}
\approx
0.890\,\Delta L^{\mathrm{pred}}-0.008.
\]

This is much cleaner than the width-sweep fit:

\[
\Delta L^{\mathrm{emp}}
\approx
0.546\,\Delta L^{\mathrm{pred}}+0.044.
\]

The remaining bias is again mostly BP fixed-kernel optimism:

\[
\mathbb E[L_{\mathrm{BP}}^{\mathrm{pred}}-L_{\mathrm{BP}}^{\mathrm{emp}}]
\approx
-0.020,
\]

with a small FA prediction bias:

\[
\mathbb E[L_{\mathrm{FA}}^{\mathrm{pred}}-L_{\mathrm{FA}}^{\mathrm{emp}}]
\approx
0.0045.
\]

## Interpretation

The earlier plot was not the cleanest validation of the theory because the
width axis mixed capacity shrinkage with finite-width NTK error. The fixed-width
N-sweep is the cleaner short-horizon validation design.

This does not yet solve the long-horizon memorization transition. Short-SGD
validates the local tangent-kernel distribution; long-horizon capacity
transition still needs either a time-varying kernel theory or a separate
alignment-dynamics theorem.