1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
|
# Oral-A-v5: layerwise causal bootstrap without KP
## Why this branch exists
The current scalable ResNet result uses reciprocal Kolen--Pollack (KP)
updates. It is therefore evidence for KP plus innovation, not evidence that
SDIL's causal vectorizer itself scales. V1--V3 tried to predict every hidden
field directly from the output error and failed in early layers. V4 supplied
the missing residual-DAG hierarchy, but used one batch-level task-loss scalar
to estimate all 267,904 feedback parameters at once. Its best early-layer
alignment stayed near zero after 800 loss queries.
V5 changes the causal information, not merely a rate or schedule. It learns
one hierarchical feedback edge at a time, starting at the readout and moving
toward the input. For each example, an independent hidden perturbation and
its own antithetic loss difference give one causal observation. Thus a batch
of 128 supplies 128 observations to one edge instead of one scalar shared by
the whole feedback network. No forward weight is read or copied, no
reciprocal KP update is used, and no reverse-mode derivative is used by the
learning rule.
This is a feasibility branch, not yet an SDIL novelty claim. If it passes,
somato-dendritic residualization must still be shown to be load-bearing on top
of the learned causal path. If it fails, current evidence does not support a
standalone scalable SDIL vectorizer.
## Update
Let `t_l` be the teaching field already available at a child population. The
feedback convolution predicts its parent field as part of the ordinary
hierarchical teaching pass. For one parent population, draw an independent
Rademacher field `xi_i` for each example and evaluate
```
d_i = [loss_i(h + sigma xi_i) - loss_i(h - sigma xi_i)] / (2 sigma)
q_i = -d_i xi_i .
```
`q_i` is an unbiased node-perturbation estimate of that example's negative
hidden gradient. BatchNorm is put in evaluation mode for these two causal
queries so examples are not coupled; ordinary task updates retain training
mode BatchNorm. If `p_l` is the full currently predicted parent teaching
field and `c_l` is the locally available child context, update only the
corresponding feedback edge:
```
Q_l <- Q_l + eta_A * Corr(q_l - p_l, c_l) / local_context_power.
```
The dense readout feedback is calibrated first with channel-constant probes.
Convolutional edges then follow reverse residual-DAG order. Recomputing the
teaching pass after every edge lets each newly calibrated child direction
serve as context for the next parent edge. Forward weights remain frozen
during this bootstrap phase.
## V5-0 mechanics gate
Before any CIFAR endpoint, deterministic tiny-network checks must show:
- evaluation-mode per-example antithetic derivatives match exact per-example
JVPs to relative error below `2e-3` at `sigma=1e-3`;
- a stored causal observation produces the same feedback update after all
forward weights are changed (absolute difference below `1e-12` in float64);
- only the selected feedback edge changes in one calibration event;
- one complete reverse sweep has finite diagnostics and all legacy
convolutional smoke checks remain green.
## V5-1 frozen-forward causal-capture gate
Use the same seed-0 ResNet-20 initialization, frozen 10,000-example
development prefix, 5,000-example validation split, and 64-example exact
gradient audit as V4. Calibration uses unaugmented batches of 128, `sigma =
0.01`, `eta_A = 0.1`, perturbation seed 5000, and 20 complete reverse sweeps.
Each sweep calibrates the readout feedback and 18 convolutional feedback
edges, for 380 edge events, 760 logical batch-loss queries, and 48,640
per-example causal observations. This is slightly below V4's 800-query
budget. Forward weights, readout, BatchNorm statistics/affines, and optimizer
state must remain bitwise fixed.
Record the matched uncalibrated HFA state and the V5 state. V5-1 passes only
if both records are finite and V5:
1. reaches early-third teaching/negative-gradient cosine at least `0.10`;
2. reaches all-layer cosine at least `0.20`;
3. improves early-third cosine over fixed HFA by at least `0.08`;
4. keeps every feedback/forward norm ratio in `[0.1, 3.0]`;
5. records exactly zero forward-weight reads in the feedback update and zero
reverse-mode learning operations.
There is no learning-rate, sweep-count, probe-shape, normalization, or
per-layer recovery grid after the endpoint. A failure closes V5.
## Conditional next gate
Only a V5-1 pass permits a separately committed 10k-example, 20-epoch
ResNet-20 task screen. It will alternate complete feedback-only reverse
cycles with forward-only task phases and will not update forward weights until
the frozen local causal-fit confidence criterion is met. Its exact cadence,
confidence threshold, learning rates, cost gate, and accuracy gate must be
fixed before any task endpoint is run.
|
