diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 14:17:04 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 14:33:50 -0500 |
| commit | 41048d9d9c399ded6f5c473b578f59e6ac375ae9 (patch) | |
| tree | 9bcfa1e540957500b9bb82e2c4e4537fe802c25f /KP_BASELINE.md | |
| parent | d761f655301e3535368a87e9d4ffc26ab1ab8510 (diff) | |
protocol: freeze reciprocal KP baseline funnel
Diffstat (limited to 'KP_BASELINE.md')
| -rw-r--r-- | KP_BASELINE.md | 78 |
1 files changed, 78 insertions, 0 deletions
diff --git a/KP_BASELINE.md b/KP_BASELINE.md new file mode 100644 index 0000000..25696f7 --- /dev/null +++ b/KP_BASELINE.md @@ -0,0 +1,78 @@ +# Modified Kolen--Pollack baseline protocol + +## Attribution and boundary + +This is the reciprocal local-plasticity mechanism of Akrout et al., *Deep +Learning without Weight Transport* (NeurIPS 2019), not an SDIL contribution. +Their equations 16--18 give forward and feedback synapses equal adjustments +from reciprocal local activity pairs and equal decay, so initially distinct +weights converge without transmitting either weight. The public author +repository is frozen for reference at revision +`688f47addd2131684da1b7829b20365f585eee66`: + +<https://github.com/makrout/Deep-Learning-without-Weight-Transport> + +The public Python implementation is a fully connected pedagogical version and +explicitly transposes a precomputed update. The implementation here instead +recomputes each reciprocal convolutional correlation from its own cached +parent activity and child teaching field. It follows the actual ResNet +residual DAG, including local ReLU and BatchNorm Jacobians and parameter-free +shortcut adjoints. The reciprocal update never reads a forward weight or a +forward update tensor. + +KP is introduced because the frozen residual-response mirror full run tests a +different, intermittent probe-response mechanism. KP is not a cadence or rate +rescue of that branch. No KP task endpoint was generated before this protocol +and its executable analyzer were committed. + +## KP-0: mechanics gate + +The convolutional smoke suite requires all of: + +- independently recomputed forward and reciprocal local directions agree to + absolute error below `1e-14` under the storage/sign convention; +- changing every W and Q value after local activities are fixed changes the + reciprocal update by exactly zero; +- exact `Q=W`, `R=-W_out^T` remains exact after two momentum-plus-decay steps; +- the public KP task step leaves all parameters graph-free; +- every pre-existing convolutional local-gradient and feedback check remains + green. + +The implemented errors are all exactly zero in float64. + +## KP-1: frozen 20-epoch useful-scale gate + +Run exactly one seed-0 ResNet-20 development record on the frozen 45k/5k +CIFAR-10 split: batch 128, standard augmentation, hidden/output LR 0.1, +epoch-100/150 step drops (therefore constant during this screen), no warmup, +momentum 0.9, weight decay `1e-4`, BatchNorm, feedback scale 1, and a 32-example +alignment audit. There is no LR, decay, initialization, or feedback-scale grid. + +The already frozen matched BP trajectory has 81.02% validation accuracy at +epoch 20 and `1.09487808e14` linearly scaled training MACs. KP-1 passes only if: + +- the record, every epoch loss, and every tracking diagnostic are finite; +- validation accuracy is at least 70% and within 15 points of BP epoch 20; +- final early-third teaching alignment is at least 0.50; +- final mean feedback/forward cosine is at least 0.80 and its epoch-11--20 + mean is at least 0.70; +- feedback learning uses zero task-loss queries; +- total MACs, including a separate correlation at every reciprocal synapse, + are at most `1.40x` matched BP. + +Failure closes KP without an extra learning-rate/decay screen. Passing opens +one full development run. + +## KP-2: conditional full baseline + +Copy KP-1 exactly for 200 epochs. KP-2 passes only if it is finite, reaches +88% validation accuracy, retains early alignment 0.80, reaches final mean +feedback/forward cosine 0.95, keeps the epoch-151--200 mean cosine at least +0.95, uses zero task-loss queries, and costs at most `1.40x` the frozen full BP +MACs. It never authorizes test access. + +KP remains an inherited baseline even if both gates pass, so it cannot raise +the reviewer score. A KP-2 pass only opens a separately frozen +raw-versus-norm-matched-raw-versus-innovation mixed-traffic experiment. The +somato-dendritic innovation must be load-bearing there to affect the SDIL +paper assessment. |
