diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-08-06 14:36:57 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-08-06 14:36:57 -0500 |
| commit | 3665a6f3821a6f35519c648efeee8a4c42c4c819 (patch) | |
| tree | a09b1ecafb4dee2dbb167f15f647165476bcd812 | |
| parent | ffaa3695da6f00a32d7b5636cff2e5faac41e6a8 (diff) | |
protocol: freeze stagewise whitened causal feedback
| -rw-r--r-- | ORAL_A_V6.md | 96 |
1 files changed, 96 insertions, 0 deletions
diff --git a/ORAL_A_V6.md b/ORAL_A_V6.md new file mode 100644 index 0000000..7663f27 --- /dev/null +++ b/ORAL_A_V6.md @@ -0,0 +1,96 @@ +# Oral-A-v6: stagewise causally whitened feedback without KP + +## Motivation and claim boundary + +V5 showed that per-example task perturbations contain useful causal +information, but its interleaved scalar-normalized delta updates failed to +carry that information into the early ResNet-20 layers. After 20 sweeps, the +readout feedback itself had parameter cosine only `0.1771`; early-third +teaching alignment was `0.00115` even though several late layers reached +`0.20--0.38`. + +One post-failure diagnostic, not a candidate result, used the same 2,560 +readout observations in a local least-squares fit. It raised last-hidden +teaching alignment to `0.8534`. This shows that the observations have enough +signal and identifies the failed online regression as a bottleneck. + +V6 is a new bounded mechanism, not a V5 rate or sweep-count recovery. It +changes both optimization and information timing: + +1. fit the readout feedback from its complete local causal dataset; +2. freeze it, collect a complete causal dataset for the last convolutional + feedback edge, and apply one diagonal-whitened local least-squares update; +3. freeze that edge and repeat toward the input. + +No parent edge observes data until its child path has been fitted. The dense +fit and convolutional sufficient statistics consume only stored task +perturbation targets, local child contexts, and the current feedback tensor. +They never read or copy a forward weight, use KP, or invoke reverse-mode +learning. V6 remains a scalable-vectorizer feasibility test, not yet the +Harnett-specific SDIL contribution. + +## Local fits + +For the readout, concatenate the per-example output signals `C` and causal +pooled-hidden targets `Y` and solve the ridge-scaled local normal equations + +``` +R^T = (C^T C + lambda I)^(-1) C^T Y, +lambda = 1e-6 trace(C^T C) / n_classes. +``` + +For convolutional edge `l`, let `e = q - p` be the difference between the +causal parent-field target and the complete current parent teaching field. +Accumulate the ordinary local correlation numerator and its per-parameter +diagonal curvature: + +``` +b_l = sum Corr(e, c_l) +d_l = sum Corr(1, c_l^2) +Q_l <- Q_l + b_l / (d_l + 1e-3 mean(d_l)). +``` + +This is a diagonal local least-squares step. The ridge is scale-relative and +fixed globally; there is no learning-rate or per-layer schedule. + +## V6-0 mechanics gate + +Before CIFAR endpoints, deterministic float64 checks must show: + +- the dense fit is invariant to changing every forward weight after causal + observations are stored, to absolute error below `1e-12`; +- the convolutional fit has the same invariance and changes only its selected + feedback edge; +- on independent Gaussian local contexts with responses produced by a known + feedback tensor, one diagonal fit reduces held-out field MSE by at least + `80%`; +- one tiny reverse stage sequence is finite and all legacy convolutional + smoke tests remain green. + +## V6-1 frozen-forward causal-capture gate + +Copy V5's seed-0 ResNet-20 state, frozen 10,000-example development prefix, +5,000-example validation split, 64-example exact-gradient audit, batch 128, +evaluation-mode causal BatchNorm, `sigma=0.01`, and perturbation seed 5000. +Use exactly 20 fixed unaugmented prefix batches for every stage. There are 19 +stages (readout plus 18 convolutional edges), 20 events per stage, 380 events, +760 logical batch-loss queries, and 48,640 per-example causal observations. +Forward/readout weights, BatchNorm state/affines, and forward optimizer state +must remain bitwise fixed. + +The only numerical constants are the readout relative ridge `1e-6` and +convolutional diagonal ridge `1e-3` written above. Record fixed HFA and the +final V6 state. V6-1 passes only if both are finite and V6: + +1. reaches early-third teaching/negative-gradient cosine at least `0.10`; +2. reaches all-layer cosine at least `0.20`; +3. improves early-third cosine over fixed HFA by at least `0.08`; +4. keeps every feedback/forward norm ratio in `[0.1, 3.0]`; +5. uses exactly the fixed event/query/observation counts and zero forward + weight reads or reverse-mode learning operations. + +There is no ridge, batch, stage-order, event-count, probe, clipping, or norm +recovery after the endpoint. Failure closes V6 and the present no-KP causal +vectorizer route. Passing opens only a separately frozen short task gate; +capture alone cannot raise the reviewer score. + |
