diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:53:02 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:53:02 -0500 |
| commit | b2ccd3aa430ecb9c267c8797ccabb3a32838f027 (patch) | |
| tree | 6934e6ea252e911020c6c96c6bdc0b94ad61c473 /RESULTS.md | |
| parent | 18aff5b55e3ad1f6e244ad25a8eae44f25f4e5c1 (diff) | |
results: pass residual mirror capture gate
Diffstat (limited to 'RESULTS.md')
| -rw-r--r-- | RESULTS.md | 25 |
1 files changed, 25 insertions, 0 deletions
@@ -828,6 +828,31 @@ gain errors and the remaining compounding angular errors can materially change the trajectory. It would be misleading to present the capture result alone as BP-equivalent learning. +### Residual response-mirror capture + +Residual response mirroring changes the local baseline's update from averaging +a fresh noisy weight estimate to predicting each child response and updating +from the local response residual. At exact symmetry each stochastic update is +zero. The frozen RRM-1 screen gives: + +| method/rate | early alignment | all-layer alignment | feedback/forward cosine | norm range | +|:--|--:|--:|--:|:--| +| fixed HFA | -0.000263 | 0.011279 | 0.001614 | [0.959, 1.055] | +| RRM 0.03 | 0.075016 | 0.235001 | 0.622965 | [0.695, 0.792] | +| RRM 0.10 | **0.665070** | **0.730253** | **0.953680** | **[0.884, 1.018]** | +| RRM 0.30 | 0.003489 | 0.053744 | 0.740396 | [0.999, 30.132] | + +Selected `eta_M=0.1` passes every frozen capture check after 20 batch-1 local +observations. It consumes zero task-loss queries and `2.4458e9` MACs. Compared +with the selected normalized mirror record, it gains 0.219 early-alignment and +0.188 all-layer-alignment points while keeping feedback norms controlled. The +failed high-rate record shows that the residual rule is not rate-insensitive: +at 0.3 the maximum norm ratio reaches 30.13 and alignment collapses. + +RRM-2 is opened using the already frozen rate and cadence. This is a strong +inherited learned-alignment baseline, not Harnett-specific evidence, so it does +not raise the SDIL paper score by itself. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`. |
