# Time-Varying Kernel Diagnostic

We tested the finite-\(T\) proposal directly.

## Setup

- task: random-label regression;
- architecture: \(16\to64\to64\to4\);
- \(N=128\);
- SGD, learning rate \(10^{-3}\);
- horizon \(T=50\);
- 2 initialization seeds;
- 4 feedback seeds per initialization;
- 8 FA trajectories total.

For each trajectory, compute three predictions:

1. empirical BP/FA train losses;
2. fixed-kernel prediction using \(K(0)\);
3. measured time-varying product using \(K(t)\) at every step.

The time-varying residual recursion is

\[
r_{t+1}^{\mathrm{tv}}
=
\left(I-\frac{\eta}{N}K_t\right)r_t^{\mathrm{tv}},
\]

with

\[
K_t^{\mathrm{BP}}=J_tJ_t^\top,
\qquad
K_t^{\mathrm{FA}}=J_t\tilde J_t^\top.
\]

No fitted scale or offset is used.

## Result

Output:

`outputs/finite_time_kernel_probe_T50_N128_8runs`

Figures:

- `outputs/finite_time_kernel_probe_T50_N128_8runs/T50_fixed_vs_timevarying_gap.png`
- `outputs/finite_time_kernel_probe_T50_N128_8runs/T50_prediction_scatter.png`

Gap prediction:

| predictor | mean gap error | gap MAE | max abs gap error |
|---|---:|---:|---:|
| fixed \(K(0)\) | 0.027683 | 0.027683 | 0.037170 |
| time-varying \(K(t)\) | 0.000273 | 0.000273 | 0.000390 |

BP loss prediction:

| predictor | mean BP error | BP MAE |
|---|---:|---:|
| fixed \(K(0)\) | -0.025404 | 0.025404 |
| time-varying \(K(t)\) | -0.000237 | 0.000237 |

FA loss prediction:

| predictor | mean FA error | FA MAE |
|---|---:|---:|
| fixed \(K(0)\) | 0.002279 | 0.004991 |
| time-varying \(K(t)\) | 0.000036 | 0.000060 |

## Interpretation

At \(T=50\), the fixed-kernel gap error is almost entirely removed by using the
measured time-varying kernel product.

Therefore, for this regime, the finite-time mismatch is dominated by kernel
drift:

\[
K_t-K_0.
\]

The second-order output Taylor residual is small at this step size/horizon,
because the measured \(K(t)\) product already matches empirical losses to
roughly \(10^{-4}\) to \(10^{-3}\) absolute error.

This validates the finite-time extension direction:

\[
\text{local theory: }K(0)
\quad\rightarrow\quad
\text{finite-time theory: }K(t).
\]

The next theoretical task is not to fit a scalar correction. It is to model the
evolution of \(K_t^{\mathrm{BP}}\) and \(K_t^{\mathrm{FA}}\), especially the FA
alignment-driven improvement of \(K_t^{\mathrm{FA}}\).