# Early-Kernel Operator Predictor

This note records the first predictive finite-\(T\) operator-level result.

## Problem

The fixed initialization tangent-kernel prediction

\[
r_{t+1}
=
\left(I-\frac{\eta}{N}K_0\right)r_t
\]

works at very short horizon, but at \(T=50\) it over-predicts the FA/BP train
gap. The time-varying diagnostic in `notes/13_time_varying_kernel_diagnostic.md`
showed that the error is caused mostly by kernel drift \(K_t-K_0\), not by a
large second-order output residual.

The goal here is a predictive, non-oracle correction: use only an early
observation window, not the final loss gap.

## Predictors

Let \(s<T\) be an early probe step. For each BP or FA rule, compute

\[
K_0^{rule},
\qquad
K_s^{rule}.
\]

The strongest predictor in this probe is the early kernel velocity model:

\[
K_t^{rule}
\approx
K_0^{rule}
+
t\frac{K_s^{rule}-K_0^{rule}}{s},
\qquad
0\le t<T.
\]

The residual prediction is then

\[
\hat r_{t+1}^{rule}
=
\left(
I-\frac{\eta}{N}\hat K_t^{rule}
\right)
\hat r_t^{rule}.
\]

No fitted scale, offset, or final-gap calibration is used.

We also tested two weaker alternatives:

- FA-only compressed interpolation:
  \[
  K_t^{FA}
  \approx
  K_0^{FA}+\alpha_t(K_0^{BP}-K_0^{FA}),
  \]
  with \(\alpha_t\) inferred from the early projection of
  \(K_s^{FA}-K_0^{FA}\).
- early re-tangent:
  run \(s\) empirical steps, then freeze \(K_s\) for the remaining \(T-s\)
  steps.

## Experiment

- task: random-label regression;
- architecture: \(16\to64\to64\to4\);
- optimizer: full-batch SGD;
- learning rate: \(10^{-3}\);
- target horizon: \(T=50\);
- early probe: \(s=20\);
- train samples:
  \[
  N\in\{64,96,128,160,192,224,256,320\};
  \]
- 4 initialization seeds and 8 feedback seeds per \(N\);
- 256 total trajectory points.

Output directory:

`outputs/compressed_operator_s20_256traj_T50_width64_plots`

Key files:

- `combined_rows.csv`
- `predictor_metrics.csv`
- `T50_s20_velocity_theory_band_empirical_trajectories.png`
- `T50_s20_empirical_trajectories_only.png`
- `T50_s20_prediction_scatter.png`
- `T50_s20_prediction_error_by_margin.png`

## Results

Across all 256 trajectory points:

| predictor | MAE | bias | corr |
|---|---:|---:|---:|
| fixed \(K(0)\) | 0.018487 | 0.018487 | 0.984460 |
| FA compressed | 0.014915 | 0.014915 | 0.990472 |
| early kernel velocity | 0.001893 | -0.001491 | 0.998848 |
| early re-tangent | 0.004868 | 0.004868 | 0.999077 |

Per-\(N\) summaries for the early kernel velocity model:

| \(N\) | hard FA margin | fixed MAE | velocity MAE |
|---:|---:|---:|---:|
| 64 | 770 | 0.025444 | 0.001621 |
| 96 | 642 | 0.021466 | 0.002578 |
| 128 | 514 | 0.020630 | 0.001886 |
| 160 | 386 | 0.017452 | 0.001879 |
| 192 | 258 | 0.016190 | 0.001556 |
| 224 | 130 | 0.015798 | 0.002354 |
| 256 | 2 | 0.015586 | 0.001548 |
| 320 | -254 | 0.015333 | 0.001725 |

## Interpretation

This is the first usable finite-\(T\) predictive version.

The fixed \(K(0)\) theory gets the ordering and rough shape right, but its
gap prediction is uniformly too high because it misses kernel drift. The early
operator-velocity model removes almost all of that bias without using the final
loss gap.

The scalar directional-gain predictor in `notes/14_early_predictor_probe.md`
failed because it compressed the dynamics too aggressively: it discarded
changes in residual direction and in the full operator. The successful object
is the operator \(K_t\), not a single scalar \(\lambda_t\).

The early re-tangent model is conservative: it remains biased high. The linear
velocity model is slightly biased low, so together they may define a clean
no-fit prediction interval around the empirical gap.

## Paper Use

This should be folded into contribution 4:

1. static distribution and local \(K(0)\) predictions;
2. finite-\(T\) failure mode from kernel drift;
3. predictive early-kernel correction;
4. trajectory-level distribution validation.

The current version is not yet a closed-form architecture-only theorem. It is a
predictive finite-time theorem conditional on the early kernel pair
\((K_0,K_s)\). That is acceptable if clearly separated from the prior-free
initialization/capacity bounds.