diff options
Diffstat (limited to 'notes/16_operator_stress_grid.md')
| -rw-r--r-- | notes/16_operator_stress_grid.md | 104 |
1 files changed, 104 insertions, 0 deletions
diff --git a/notes/16_operator_stress_grid.md b/notes/16_operator_stress_grid.md new file mode 100644 index 0000000..7d7bf25 --- /dev/null +++ b/notes/16_operator_stress_grid.md @@ -0,0 +1,104 @@ +# Operator Predictor Stress Grid + +This note tests whether the early-kernel operator predictor from +`notes/15_early_kernel_operator_predictor.md` survives changes in network +depth, width, and training horizon. + +## Predictor + +For each rule, use the early operator velocity + +\[ +\hat K_t^{rule} += +K_0^{rule} ++ +t\frac{K_s^{rule}-K_0^{rule}}{s}, +\qquad 0\le t<T, +\] + +and roll out + +\[ +\hat r_{t+1}^{rule} += +\left(I-\frac{\eta}{N}\hat K_t^{rule}\right) +\hat r_t^{rule}. +\] + +This uses \((K_0,K_s)\) only. There is no fitted scale, offset, or final-gap +calibration. + +## Grid + +All experiments use random-label regression, input dimension 16, output +dimension 4, full-batch SGD, and learning rate \(10^{-3}\). + +Stress settings: + +- depth: \(1,2,3\) hidden layers at width 64, \(T=50,s=20\); +- width: \(32,64,96\) at depth 2, \(T=50,s=20\); +- horizon: \(T=25,s=10\), \(T=50,s=20\), \(T=100,s=40\) at depth 2, width 64. + +Each new stress setting uses 5 train-sample values around the hard FA capacity +margin and 8 trajectories per value. The baseline \(d=2,w=64,T=50,s=20\) uses +the existing 256 trajectory points. + +Output directory: + +`outputs/operator_stress_grid_summary` + +Key files: + +- `stress_grid_rows.csv` +- `stress_grid_metrics.csv` +- `stress_grid_gap_mae_by_setting.png` +- `stress_grid_linear_bias_by_margin.png` +- `stress_grid_linear_prediction_scatter.png` + +## Results + +| setting | rows | fixed MAE | linear MAE | retangent MAE | linear bias | corr | +|---|---:|---:|---:|---:|---:|---:| +| \(d=1,w=64,T=50,s=20\) | 40 | 0.003862 | 0.000233 | 0.001162 | -0.000233 | 0.999891 | +| \(d=2,w=32,T=50,s=20\) | 40 | 0.047110 | 0.005652 | 0.013333 | -0.005375 | 0.994077 | +| \(d=2,w=64,T=25,s=10\) | 40 | 0.020808 | 0.002722 | 0.005084 | -0.002668 | 0.999074 | +| \(d=2,w=64,T=50,s=20\) | 256 | 0.018487 | 0.001893 | 0.004868 | -0.001491 | 0.998848 | +| \(d=2,w=64,T=100,s=40\) | 40 | 0.034378 | 0.002647 | 0.008100 | -0.002169 | 0.998653 | +| \(d=2,w=96,T=50,s=20\) | 40 | 0.016912 | 0.001794 | 0.004004 | -0.001551 | 0.998634 | +| \(d=3,w=64,T=50,s=20\) | 40 | 0.051670 | 0.005049 | 0.012450 | -0.004916 | 0.998546 | + +## Interpretation + +The predictor holds across all tested depth, width, and horizon changes. + +The hard cases are the narrower and deeper MLPs: + +- width 32 increases finite-width/kernel-drift error; +- depth 3 increases the magnitude of kernel drift and FA/BP mismatch. + +Even in these cases, the early operator-velocity predictor reduces the fixed +\(K(0)\) gap error by roughly one order of magnitude. The remaining systematic +error is a mild negative bias, meaning the linear velocity model slightly +over-extrapolates kernel drift. + +The early re-tangent predictor is consistently conservative and biased high. +Together, early re-tangent and linear velocity form a useful no-fit bracket: + +\[ +\Delta L_T^{linear} +\lesssim +\Delta L_T^{empirical} +\lesssim +\Delta L_T^{retangent} +\] + +in most tested settings. + +## Current Boundary + +This stress grid supports the finite-\(T\) predictor conditional on early +operator observations \((K_0,K_s)\). It does not yet prove an architecture-only +closed-form predictor for long training. For that stronger claim, the next +theoretical object would need to predict \(K_s-K_0\) from initialization +statistics rather than measuring \(K_s\). |
