diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:25:15 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:25:15 -0500 |
| commit | d4a521072a25d8afdd62021f43538d7dad7586fd (patch) | |
| tree | c6d2dc1b6bdb9c857970067473e5df45906dbdec /ORAL_A_V4.md | |
| parent | c24a6a575bce15e7d7da541ecb752c65b5e68e47 (diff) | |
protocol: freeze hierarchical causal-capture funnel
Diffstat (limited to 'ORAL_A_V4.md')
| -rw-r--r-- | ORAL_A_V4.md | 94 |
1 files changed, 94 insertions, 0 deletions
diff --git a/ORAL_A_V4.md b/ORAL_A_V4.md new file mode 100644 index 0000000..db1af1a --- /dev/null +++ b/ORAL_A_V4.md @@ -0,0 +1,94 @@ +# Oral-A-v4: causal calibration of hierarchical feedback maps + +## Status and claim boundary + +The output-error-only V2/V3 feedback families failed their frozen early-layer +causal-capture gates. A held-out oracle then showed that ordinary downstream +activations are not enough, whereas gated 3x3 maps of the actual residual-DAG +child error fields can reconstruct early credit almost exactly. Fixed random +hierarchical FA subsequently improved the matched short ResNet-20 endpoint to +43.52% from DFA's 37.16%, but failed its frozen 50% full-run gate. + +V4 is the first trainable use of that diagnosed spatial hierarchy. Every +feedback convolution consumes its local child teaching field. One independent +Rademacher tensor is drawn in each feedback-parameter space; all induced parent +fields are injected in a single antithetic pair, and the scalar task-loss +derivative estimates each edge's causal target moment. The locally computed +prediction moment is subtracted before updating Q/R. Forward weights are never +copied or read by the learning rule, and no reverse-mode differentiation is +used. + +Recursive learned feedback is not claimed as the Harnett-specific novelty. +V4 is an engineering prerequisite for a later hierarchical innovation model +and must be compared with fixed HFA, learned FA/weight mirroring, and BurstCCN. +The frozen Oral-A-v1 failure remains failed; V4 cannot retroactively open A4. +No V4 GPU endpoint was observed before this protocol and its selector were +committed. CIFAR-10 test and confirmation seeds 10--14 remain untouched. + +## V4-0: mechanics gate + +Before any endpoint, `experiments/conv_local_smoke.py` must verify: + +- the simultaneous BatchNorm-coupled antithetic JVP to relative error below + `2e-6`; +- under an audit-only symmetric Q=W and R=-Wout-transpose copy, every predicted + local feedback moment equals its exact causal delta-rule target to relative + error below `2e-12`; +- the symmetric hierarchical field and local parameter update continue to + reproduce exact BP under the pre-existing thresholds; +- all legacy convolutional checks remain green. + +This gate passed with JVP error `6.16e-10`, delta-rule relative error +`2.30e-16`, hidden-field relative error `4.97e-16`, and parameter-update +absolute error `1.49e-8`. Actual training never makes the symmetric copy. + +## V4-1: frozen-forward causal-capture gate + +Use seed-0 ResNet-20, the first 10,000 development-training examples, batch +128, the frozen 45k/5k split, random hierarchical feedback scale 1, one +direction, `sigma=0.01`, perturbation seed 1000, and a fixed 64-example exact +gradient audit. Forward weights, readout, BatchNorm state, and affine +parameters remain bitwise fixed. + +Record one uncalibrated HFA reference and calibrate Q/R for 400 feedback-only +minibatches at `eta_A in {0.1, 1.0, 10.0}`. Select maximum early-third +teaching/negative-gradient cosine, then maximum all-layer cosine, then lower +rate. V4-1 passes only if all four records are finite and the selected method: + +1. reaches early-third alignment at least `0.05`; +2. reaches all-layer alignment at least `0.10`; +3. exceeds the matched uncalibrated HFA early alignment by at least `0.04`; +4. keeps every feedback/forward audit norm ratio in `[0.1, 3.0]`. + +The norm gate is audit-only and prevents a scale explosion from being hidden +by cosine. No extra rate, query count, initialization, warmup, normalization, +or per-layer schedule is added after endpoints are observed. + +## V4-2: bounded short accuracy gate + +Only a V4-1 pass opens a 10k-example, 20-epoch ResNet-20 validation screen. +Copy A2b exactly: batch 128, cosine decay, no warmup, momentum 0.9, weight decay +`1e-4`, output LR 0.1, and hidden LR `{0.03, 0.1}`. Use the V4-1-selected +feedback rate, 400 feedback-only warmup minibatches, then K1/every-4 joint +calibration. Select validation accuracy, then lower MACs/rate. + +Advance only if both runs are finite, selected V4 reaches at least 60%, exceeds +fixed HFA's 43.52% by at least 10 points, retains early alignment at least +0.05, and does not exceed `1.5x` exact-BP estimated MACs. Failure closes V4 +without a full run or an expanded grid. + +## V4-3: full validation and confirmation rule + +Only a V4-2 pass opens one 200-epoch seed-0 validation run on all 45,000 +development-training examples. Copy the selected hidden rate, output rate 0.1, +400-step feedback warmup, K1/every-4 joint calibration, and the A1 step drops +at epochs 100/150. It must be finite, finish within 5 points of BP 91.62%, beat +DFA 33.06% and fixed HFA 43.52% by at least 2 points, retain early alignment +0.05, and remain within `1.5x` BP MACs. There is no recovery branch. + +A complete V4-3 validation pass may move the strict reviewer forecast from 5 +to 6 and permits a new, separately frozen hierarchical-SDIL residualization +test. It does not itself establish Harnett-specific novelty. Only after that +innovation test passes may an independent ResNet-20/32/56 confirmation be +specified; no test endpoint is authorized by V4-1 or V4-2. + |
