Stress test early-kernel operator predictor

1 files changed, 104 insertions, 0 deletions

diff --git a/notes/16_operator_stress_grid.md b/notes/16_operator_stress_grid.md
new file mode 100644
index 0000000..7d7bf25
--- /dev/null
+++ b/notes/16_operator_stress_grid.md
@@ -0,0 +1,104 @@
+# Operator Predictor Stress Grid
+
+This note tests whether the early-kernel operator predictor from
+`notes/15_early_kernel_operator_predictor.md` survives changes in network
+depth, width, and training horizon.
+
+## Predictor
+
+For each rule, use the early operator velocity
+
+\[
+\hat K_t^{rule}
+=
+K_0^{rule}
++
+t\frac{K_s^{rule}-K_0^{rule}}{s},
+\qquad 0\le t<T,
+\]
+
+and roll out
+
+\[
+\hat r_{t+1}^{rule}
+=
+\left(I-\frac{\eta}{N}\hat K_t^{rule}\right)
+\hat r_t^{rule}.
+\]
+
+This uses \((K_0,K_s)\) only. There is no fitted scale, offset, or final-gap
+calibration.
+
+## Grid
+
+All experiments use random-label regression, input dimension 16, output
+dimension 4, full-batch SGD, and learning rate \(10^{-3}\).
+
+Stress settings:
+
+- depth: \(1,2,3\) hidden layers at width 64, \(T=50,s=20\);
+- width: \(32,64,96\) at depth 2, \(T=50,s=20\);
+- horizon: \(T=25,s=10\), \(T=50,s=20\), \(T=100,s=40\) at depth 2, width 64.
+
+Each new stress setting uses 5 train-sample values around the hard FA capacity
+margin and 8 trajectories per value. The baseline \(d=2,w=64,T=50,s=20\) uses
+the existing 256 trajectory points.
+
+Output directory:
+
+`outputs/operator_stress_grid_summary`
+
+Key files:
+
+- `stress_grid_rows.csv`
+- `stress_grid_metrics.csv`
+- `stress_grid_gap_mae_by_setting.png`
+- `stress_grid_linear_bias_by_margin.png`
+- `stress_grid_linear_prediction_scatter.png`
+
+## Results
+
+| setting | rows | fixed MAE | linear MAE | retangent MAE | linear bias | corr |
+|---|---:|---:|---:|---:|---:|---:|
+| \(d=1,w=64,T=50,s=20\) | 40 | 0.003862 | 0.000233 | 0.001162 | -0.000233 | 0.999891 |
+| \(d=2,w=32,T=50,s=20\) | 40 | 0.047110 | 0.005652 | 0.013333 | -0.005375 | 0.994077 |
+| \(d=2,w=64,T=25,s=10\) | 40 | 0.020808 | 0.002722 | 0.005084 | -0.002668 | 0.999074 |
+| \(d=2,w=64,T=50,s=20\) | 256 | 0.018487 | 0.001893 | 0.004868 | -0.001491 | 0.998848 |
+| \(d=2,w=64,T=100,s=40\) | 40 | 0.034378 | 0.002647 | 0.008100 | -0.002169 | 0.998653 |
+| \(d=2,w=96,T=50,s=20\) | 40 | 0.016912 | 0.001794 | 0.004004 | -0.001551 | 0.998634 |
+| \(d=3,w=64,T=50,s=20\) | 40 | 0.051670 | 0.005049 | 0.012450 | -0.004916 | 0.998546 |
+
+## Interpretation
+
+The predictor holds across all tested depth, width, and horizon changes.
+
+The hard cases are the narrower and deeper MLPs:
+
+- width 32 increases finite-width/kernel-drift error;
+- depth 3 increases the magnitude of kernel drift and FA/BP mismatch.
+
+Even in these cases, the early operator-velocity predictor reduces the fixed
+\(K(0)\) gap error by roughly one order of magnitude. The remaining systematic
+error is a mild negative bias, meaning the linear velocity model slightly
+over-extrapolates kernel drift.
+
+The early re-tangent predictor is consistently conservative and biased high.
+Together, early re-tangent and linear velocity form a useful no-fit bracket:
+
+\[
+\Delta L_T^{linear}
+\lesssim
+\Delta L_T^{empirical}
+\lesssim
+\Delta L_T^{retangent}
+\]
+
+in most tested settings.
+
+## Current Boundary
+
+This stress grid supports the finite-\(T\) predictor conditional on early
+operator observations \((K_0,K_s)\). It does not yet prove an architecture-only
+closed-form predictor for long training. For that stronger claim, the next
+theoretical object would need to predict \(K_s-K_0\) from initialization
+statistics rather than measuring \(K_s\).