summaryrefslogtreecommitdiff
path: root/ORAL_A_V5.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-08-06 14:29:27 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-08-06 14:29:27 -0500
commit2eacd6a71aaf3793045cd953c47678ceec23b1d3 (patch)
treebc630a26d649f8a70cafe369deac7da8f779690b /ORAL_A_V5.md
parent09470760039d937575128d62b0d32db7df518f45 (diff)
protocol: freeze no-KP layerwise causal bootstrap
Diffstat (limited to 'ORAL_A_V5.md')
-rw-r--r--ORAL_A_V5.md101
1 files changed, 101 insertions, 0 deletions
diff --git a/ORAL_A_V5.md b/ORAL_A_V5.md
new file mode 100644
index 0000000..c00e751
--- /dev/null
+++ b/ORAL_A_V5.md
@@ -0,0 +1,101 @@
+# Oral-A-v5: layerwise causal bootstrap without KP
+
+## Why this branch exists
+
+The current scalable ResNet result uses reciprocal Kolen--Pollack (KP)
+updates. It is therefore evidence for KP plus innovation, not evidence that
+SDIL's causal vectorizer itself scales. V1--V3 tried to predict every hidden
+field directly from the output error and failed in early layers. V4 supplied
+the missing residual-DAG hierarchy, but used one batch-level task-loss scalar
+to estimate all 267,904 feedback parameters at once. Its best early-layer
+alignment stayed near zero after 800 loss queries.
+
+V5 changes the causal information, not merely a rate or schedule. It learns
+one hierarchical feedback edge at a time, starting at the readout and moving
+toward the input. For each example, an independent hidden perturbation and
+its own antithetic loss difference give one causal observation. Thus a batch
+of 128 supplies 128 observations to one edge instead of one scalar shared by
+the whole feedback network. No forward weight is read or copied, no
+reciprocal KP update is used, and no reverse-mode derivative is used by the
+learning rule.
+
+This is a feasibility branch, not yet an SDIL novelty claim. If it passes,
+somato-dendritic residualization must still be shown to be load-bearing on top
+of the learned causal path. If it fails, current evidence does not support a
+standalone scalable SDIL vectorizer.
+
+## Update
+
+Let `t_l` be the teaching field already available at a child population. The
+feedback convolution predicts its parent field as part of the ordinary
+hierarchical teaching pass. For one parent population, draw an independent
+Rademacher field `xi_i` for each example and evaluate
+
+```
+d_i = [loss_i(h + sigma xi_i) - loss_i(h - sigma xi_i)] / (2 sigma)
+q_i = -d_i xi_i .
+```
+
+`q_i` is an unbiased node-perturbation estimate of that example's negative
+hidden gradient. BatchNorm is put in evaluation mode for these two causal
+queries so examples are not coupled; ordinary task updates retain training
+mode BatchNorm. If `p_l` is the full currently predicted parent teaching
+field and `c_l` is the locally available child context, update only the
+corresponding feedback edge:
+
+```
+Q_l <- Q_l + eta_A * Corr(q_l - p_l, c_l) / local_context_power.
+```
+
+The dense readout feedback is calibrated first with channel-constant probes.
+Convolutional edges then follow reverse residual-DAG order. Recomputing the
+teaching pass after every edge lets each newly calibrated child direction
+serve as context for the next parent edge. Forward weights remain frozen
+during this bootstrap phase.
+
+## V5-0 mechanics gate
+
+Before any CIFAR endpoint, deterministic tiny-network checks must show:
+
+- evaluation-mode per-example antithetic derivatives match exact per-example
+ JVPs to relative error below `2e-3` at `sigma=1e-3`;
+- a stored causal observation produces the same feedback update after all
+ forward weights are changed (absolute difference below `1e-12` in float64);
+- only the selected feedback edge changes in one calibration event;
+- one complete reverse sweep has finite diagnostics and all legacy
+ convolutional smoke checks remain green.
+
+## V5-1 frozen-forward causal-capture gate
+
+Use the same seed-0 ResNet-20 initialization, frozen 10,000-example
+development prefix, 5,000-example validation split, and 64-example exact
+gradient audit as V4. Calibration uses unaugmented batches of 128, `sigma =
+0.01`, `eta_A = 0.1`, perturbation seed 5000, and 20 complete reverse sweeps.
+Each sweep calibrates the readout feedback and 18 convolutional feedback
+edges, for 380 edge events, 760 logical batch-loss queries, and 48,640
+per-example causal observations. This is slightly below V4's 800-query
+budget. Forward weights, readout, BatchNorm statistics/affines, and optimizer
+state must remain bitwise fixed.
+
+Record the matched uncalibrated HFA state and the V5 state. V5-1 passes only
+if both records are finite and V5:
+
+1. reaches early-third teaching/negative-gradient cosine at least `0.10`;
+2. reaches all-layer cosine at least `0.20`;
+3. improves early-third cosine over fixed HFA by at least `0.08`;
+4. keeps every feedback/forward norm ratio in `[0.1, 3.0]`;
+5. records exactly zero forward-weight reads in the feedback update and zero
+ reverse-mode learning operations.
+
+There is no learning-rate, sweep-count, probe-shape, normalization, or
+per-layer recovery grid after the endpoint. A failure closes V5.
+
+## Conditional next gate
+
+Only a V5-1 pass permits a separately committed 10k-example, 20-epoch
+ResNet-20 task screen. It will alternate complete feedback-only reverse
+cycles with forward-only task phases and will not update forward weights until
+the frozen local causal-fit confidence criterion is met. Its exact cadence,
+confidence threshold, learning rates, cost gate, and accuracy gate must be
+fixed before any task endpoint is run.
+