blob: 19cef0a98e8c8e0bd8e32c8cc54f916cb89be750 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
|
# Time-Varying Kernel Diagnostic
We tested the finite-\(T\) proposal directly.
## Setup
- task: random-label regression;
- architecture: \(16\to64\to64\to4\);
- \(N=128\);
- SGD, learning rate \(10^{-3}\);
- horizon \(T=50\);
- 2 initialization seeds;
- 4 feedback seeds per initialization;
- 8 FA trajectories total.
For each trajectory, compute three predictions:
1. empirical BP/FA train losses;
2. fixed-kernel prediction using \(K(0)\);
3. measured time-varying product using \(K(t)\) at every step.
The time-varying residual recursion is
\[
r_{t+1}^{\mathrm{tv}}
=
\left(I-\frac{\eta}{N}K_t\right)r_t^{\mathrm{tv}},
\]
with
\[
K_t^{\mathrm{BP}}=J_tJ_t^\top,
\qquad
K_t^{\mathrm{FA}}=J_t\tilde J_t^\top.
\]
No fitted scale or offset is used.
## Result
Output:
`outputs/finite_time_kernel_probe_T50_N128_8runs`
Figures:
- `outputs/finite_time_kernel_probe_T50_N128_8runs/T50_fixed_vs_timevarying_gap.png`
- `outputs/finite_time_kernel_probe_T50_N128_8runs/T50_prediction_scatter.png`
Gap prediction:
| predictor | mean gap error | gap MAE | max abs gap error |
|---|---:|---:|---:|
| fixed \(K(0)\) | 0.027683 | 0.027683 | 0.037170 |
| time-varying \(K(t)\) | 0.000273 | 0.000273 | 0.000390 |
BP loss prediction:
| predictor | mean BP error | BP MAE |
|---|---:|---:|
| fixed \(K(0)\) | -0.025404 | 0.025404 |
| time-varying \(K(t)\) | -0.000237 | 0.000237 |
FA loss prediction:
| predictor | mean FA error | FA MAE |
|---|---:|---:|
| fixed \(K(0)\) | 0.002279 | 0.004991 |
| time-varying \(K(t)\) | 0.000036 | 0.000060 |
## Interpretation
At \(T=50\), the fixed-kernel gap error is almost entirely removed by using the
measured time-varying kernel product.
Therefore, for this regime, the finite-time mismatch is dominated by kernel
drift:
\[
K_t-K_0.
\]
The second-order output Taylor residual is small at this step size/horizon,
because the measured \(K(t)\) product already matches empirical losses to
roughly \(10^{-4}\) to \(10^{-3}\) absolute error.
This validates the finite-time extension direction:
\[
\text{local theory: }K(0)
\quad\rightarrow\quad
\text{finite-time theory: }K(t).
\]
The next theoretical task is not to fit a scalar correction. It is to model the
evolution of \(K_t^{\mathrm{BP}}\) and \(K_t^{\mathrm{FA}}\), especially the FA
alignment-driven improvement of \(K_t^{\mathrm{FA}}\).
|
