1 files changed, 98 insertions, 0 deletions
diff --git a/notes/13_time_varying_kernel_diagnostic.md b/notes/13_time_varying_kernel_diagnostic.md
new file mode 100644
index 0000000..19cef0a
--- /dev/null
+++ b/notes/13_time_varying_kernel_diagnostic.md
@@ -0,0 +1,98 @@
+# Time-Varying Kernel Diagnostic
+
+We tested the finite-\(T\) proposal directly.
+
+## Setup
+
+- task: random-label regression;
+- architecture: \(16\to64\to64\to4\);
+- \(N=128\);
+- SGD, learning rate \(10^{-3}\);
+- horizon \(T=50\);
+- 2 initialization seeds;
+- 4 feedback seeds per initialization;
+- 8 FA trajectories total.
+
+For each trajectory, compute three predictions:
+
+1. empirical BP/FA train losses;
+2. fixed-kernel prediction using \(K(0)\);
+3. measured time-varying product using \(K(t)\) at every step.
+
+The time-varying residual recursion is
+
+\[
+r_{t+1}^{\mathrm{tv}}
+=
+\left(I-\frac{\eta}{N}K_t\right)r_t^{\mathrm{tv}},
+\]
+
+with
+
+\[
+K_t^{\mathrm{BP}}=J_tJ_t^\top,
+\qquad
+K_t^{\mathrm{FA}}=J_t\tilde J_t^\top.
+\]
+
+No fitted scale or offset is used.
+
+## Result
+
+Output:
+
+`outputs/finite_time_kernel_probe_T50_N128_8runs`
+
+Figures:
+
+- `outputs/finite_time_kernel_probe_T50_N128_8runs/T50_fixed_vs_timevarying_gap.png`
+- `outputs/finite_time_kernel_probe_T50_N128_8runs/T50_prediction_scatter.png`
+
+Gap prediction:
+
+| predictor | mean gap error | gap MAE | max abs gap error |
+|---|---:|---:|---:|
+| fixed \(K(0)\) | 0.027683 | 0.027683 | 0.037170 |
+| time-varying \(K(t)\) | 0.000273 | 0.000273 | 0.000390 |
+
+BP loss prediction:
+
+| predictor | mean BP error | BP MAE |
+|---|---:|---:|
+| fixed \(K(0)\) | -0.025404 | 0.025404 |
+| time-varying \(K(t)\) | -0.000237 | 0.000237 |
+
+FA loss prediction:
+
+| predictor | mean FA error | FA MAE |
+|---|---:|---:|
+| fixed \(K(0)\) | 0.002279 | 0.004991 |
+| time-varying \(K(t)\) | 0.000036 | 0.000060 |
+
+## Interpretation
+
+At \(T=50\), the fixed-kernel gap error is almost entirely removed by using the
+measured time-varying kernel product.
+
+Therefore, for this regime, the finite-time mismatch is dominated by kernel
+drift:
+
+\[
+K_t-K_0.
+\]
+
+The second-order output Taylor residual is small at this step size/horizon,
+because the measured \(K(t)\) product already matches empirical losses to
+roughly \(10^{-4}\) to \(10^{-3}\) absolute error.
+
+This validates the finite-time extension direction:
+
+\[
+\text{local theory: }K(0)
+\quad\rightarrow\quad
+\text{finite-time theory: }K(t).
+\]
+
+The next theoretical task is not to fit a scalar correction. It is to model the
+evolution of \(K_t^{\mathrm{BP}}\) and \(K_t^{\mathrm{FA}}\), especially the FA
+alignment-driven improvement of \(K_t^{\mathrm{FA}}\).