From 2eacd6a71aaf3793045cd953c47678ceec23b1d3 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Thu, 6 Aug 2026 14:29:27 -0500 Subject: protocol: freeze no-KP layerwise causal bootstrap --- ORAL_A_V5.md | 101 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 ORAL_A_V5.md (limited to 'ORAL_A_V5.md') diff --git a/ORAL_A_V5.md b/ORAL_A_V5.md new file mode 100644 index 0000000..c00e751 --- /dev/null +++ b/ORAL_A_V5.md @@ -0,0 +1,101 @@ +# Oral-A-v5: layerwise causal bootstrap without KP + +## Why this branch exists + +The current scalable ResNet result uses reciprocal Kolen--Pollack (KP) +updates. It is therefore evidence for KP plus innovation, not evidence that +SDIL's causal vectorizer itself scales. V1--V3 tried to predict every hidden +field directly from the output error and failed in early layers. V4 supplied +the missing residual-DAG hierarchy, but used one batch-level task-loss scalar +to estimate all 267,904 feedback parameters at once. Its best early-layer +alignment stayed near zero after 800 loss queries. + +V5 changes the causal information, not merely a rate or schedule. It learns +one hierarchical feedback edge at a time, starting at the readout and moving +toward the input. For each example, an independent hidden perturbation and +its own antithetic loss difference give one causal observation. Thus a batch +of 128 supplies 128 observations to one edge instead of one scalar shared by +the whole feedback network. No forward weight is read or copied, no +reciprocal KP update is used, and no reverse-mode derivative is used by the +learning rule. + +This is a feasibility branch, not yet an SDIL novelty claim. If it passes, +somato-dendritic residualization must still be shown to be load-bearing on top +of the learned causal path. If it fails, current evidence does not support a +standalone scalable SDIL vectorizer. + +## Update + +Let `t_l` be the teaching field already available at a child population. The +feedback convolution predicts its parent field as part of the ordinary +hierarchical teaching pass. For one parent population, draw an independent +Rademacher field `xi_i` for each example and evaluate + +``` +d_i = [loss_i(h + sigma xi_i) - loss_i(h - sigma xi_i)] / (2 sigma) +q_i = -d_i xi_i . +``` + +`q_i` is an unbiased node-perturbation estimate of that example's negative +hidden gradient. BatchNorm is put in evaluation mode for these two causal +queries so examples are not coupled; ordinary task updates retain training +mode BatchNorm. If `p_l` is the full currently predicted parent teaching +field and `c_l` is the locally available child context, update only the +corresponding feedback edge: + +``` +Q_l <- Q_l + eta_A * Corr(q_l - p_l, c_l) / local_context_power. +``` + +The dense readout feedback is calibrated first with channel-constant probes. +Convolutional edges then follow reverse residual-DAG order. Recomputing the +teaching pass after every edge lets each newly calibrated child direction +serve as context for the next parent edge. Forward weights remain frozen +during this bootstrap phase. + +## V5-0 mechanics gate + +Before any CIFAR endpoint, deterministic tiny-network checks must show: + +- evaluation-mode per-example antithetic derivatives match exact per-example + JVPs to relative error below `2e-3` at `sigma=1e-3`; +- a stored causal observation produces the same feedback update after all + forward weights are changed (absolute difference below `1e-12` in float64); +- only the selected feedback edge changes in one calibration event; +- one complete reverse sweep has finite diagnostics and all legacy + convolutional smoke checks remain green. + +## V5-1 frozen-forward causal-capture gate + +Use the same seed-0 ResNet-20 initialization, frozen 10,000-example +development prefix, 5,000-example validation split, and 64-example exact +gradient audit as V4. Calibration uses unaugmented batches of 128, `sigma = +0.01`, `eta_A = 0.1`, perturbation seed 5000, and 20 complete reverse sweeps. +Each sweep calibrates the readout feedback and 18 convolutional feedback +edges, for 380 edge events, 760 logical batch-loss queries, and 48,640 +per-example causal observations. This is slightly below V4's 800-query +budget. Forward weights, readout, BatchNorm statistics/affines, and optimizer +state must remain bitwise fixed. + +Record the matched uncalibrated HFA state and the V5 state. V5-1 passes only +if both records are finite and V5: + +1. reaches early-third teaching/negative-gradient cosine at least `0.10`; +2. reaches all-layer cosine at least `0.20`; +3. improves early-third cosine over fixed HFA by at least `0.08`; +4. keeps every feedback/forward norm ratio in `[0.1, 3.0]`; +5. records exactly zero forward-weight reads in the feedback update and zero + reverse-mode learning operations. + +There is no learning-rate, sweep-count, probe-shape, normalization, or +per-layer recovery grid after the endpoint. A failure closes V5. + +## Conditional next gate + +Only a V5-1 pass permits a separately committed 10k-example, 20-epoch +ResNet-20 task screen. It will alternate complete feedback-only reverse +cycles with forward-only task phases and will not update forward weights until +the frozen local causal-fit confidence criterion is met. Its exact cadence, +confidence threshold, learning rates, cost gate, and accuracy gate must be +fixed before any task endpoint is run. + -- cgit v1.2.3