1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
# Operator Predictor Stress Grid
This note tests whether the early-kernel operator predictor from
`notes/15_early_kernel_operator_predictor.md` survives changes in network
depth, width, and training horizon.
## Predictor
For each rule, use the early operator velocity
\[
\hat K_t^{rule}
=
K_0^{rule}
+
t\frac{K_s^{rule}-K_0^{rule}}{s},
\qquad 0\le t<T,
\]
and roll out
\[
\hat r_{t+1}^{rule}
=
\left(I-\frac{\eta}{N}\hat K_t^{rule}\right)
\hat r_t^{rule}.
\]
This uses \((K_0,K_s)\) only. There is no fitted scale, offset, or final-gap
calibration.
## Grid
All experiments use random-label regression, input dimension 16, output
dimension 4, full-batch SGD, and learning rate \(10^{-3}\).
Stress settings:
- depth: \(1,2,3\) hidden layers at width 64, \(T=50,s=20\);
- width: \(32,64,96\) at depth 2, \(T=50,s=20\);
- horizon: \(T=25,s=10\), \(T=50,s=20\), \(T=100,s=40\) at depth 2, width 64.
Each new stress setting uses 5 train-sample values around the hard FA capacity
margin and 8 trajectories per value. The baseline \(d=2,w=64,T=50,s=20\) uses
the existing 256 trajectory points.
Output directory:
`outputs/operator_stress_grid_summary`
Key files:
- `stress_grid_rows.csv`
- `stress_grid_metrics.csv`
- `stress_grid_gap_mae_by_setting.png`
- `stress_grid_linear_bias_by_margin.png`
- `stress_grid_linear_prediction_scatter.png`
## Results
| setting | rows | fixed MAE | linear MAE | retangent MAE | linear bias | corr |
|---|---:|---:|---:|---:|---:|---:|
| \(d=1,w=64,T=50,s=20\) | 40 | 0.003862 | 0.000233 | 0.001162 | -0.000233 | 0.999891 |
| \(d=2,w=32,T=50,s=20\) | 40 | 0.047110 | 0.005652 | 0.013333 | -0.005375 | 0.994077 |
| \(d=2,w=64,T=25,s=10\) | 40 | 0.020808 | 0.002722 | 0.005084 | -0.002668 | 0.999074 |
| \(d=2,w=64,T=50,s=20\) | 256 | 0.018487 | 0.001893 | 0.004868 | -0.001491 | 0.998848 |
| \(d=2,w=64,T=100,s=40\) | 40 | 0.034378 | 0.002647 | 0.008100 | -0.002169 | 0.998653 |
| \(d=2,w=96,T=50,s=20\) | 40 | 0.016912 | 0.001794 | 0.004004 | -0.001551 | 0.998634 |
| \(d=3,w=64,T=50,s=20\) | 40 | 0.051670 | 0.005049 | 0.012450 | -0.004916 | 0.998546 |
## Interpretation
The predictor holds across all tested depth, width, and horizon changes.
The hard cases are the narrower and deeper MLPs:
- width 32 increases finite-width/kernel-drift error;
- depth 3 increases the magnitude of kernel drift and FA/BP mismatch.
Even in these cases, the early operator-velocity predictor reduces the fixed
\(K(0)\) gap error by roughly one order of magnitude. The remaining systematic
error is a mild negative bias, meaning the linear velocity model slightly
over-extrapolates kernel drift.
The early re-tangent predictor is consistently conservative and biased high.
Together, early re-tangent and linear velocity form a useful no-fit bracket:
\[
\Delta L_T^{linear}
\lesssim
\Delta L_T^{empirical}
\lesssim
\Delta L_T^{retangent}
\]
in most tested settings.
## Current Boundary
This stress grid supports the finite-\(T\) predictor conditional on early
operator observations \((K_0,K_s)\). It does not yet prove an architecture-only
closed-form predictor for long training. For that stronger claim, the next
theoretical object would need to predict \(K_s-K_0\) from initialization
statistics rather than measuring \(K_s\).
|
