diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:12:33 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:12:33 -0500 |
| commit | f37644eb452f0eb3d7f368740f8b39493f11b2aa (patch) | |
| tree | a9030326db03e06e1b21ee1d8bcf1f378fc02fc4 /HFA_BASELINE.md | |
| parent | 7ca9658999c8f882b3d5d295a4201cc9a1ce9cde (diff) | |
protocol: freeze convolutional HFA baseline funnel
Diffstat (limited to 'HFA_BASELINE.md')
| -rw-r--r-- | HFA_BASELINE.md | 64 |
1 files changed, 64 insertions, 0 deletions
diff --git a/HFA_BASELINE.md b/HFA_BASELINE.md new file mode 100644 index 0000000..1dc5f87 --- /dev/null +++ b/HFA_BASELINE.md @@ -0,0 +1,64 @@ +# Convolutional HFA baseline protocol + +## Status and claim boundary + +This protocol was frozen before observing any convolutional hierarchical-FA +accuracy endpoint. It adds a matched local-learning baseline after the +hierarchical oracle localized the ResNet bottleneck to spatial child-error +fields. HFA is not an SDIL novelty claim: recursive random feedback is prior +art, and a strong HFA result cannot rescue the failed frozen Oral-A A3 gate or +open its sealed A4 test panel. + +The implementation follows the actual option-A residual DAG. Every learned +forward convolution has an independent fixed 3x3 feedback tensor; shortcut +adjoints are parameter-free, and ReLU and BatchNorm Jacobians are local. The +feedback code never reads a downstream forward convolution. In an audit-only +symmetric test, setting the feedback tensors to the forward tensors and the +readout feedback to the negative readout transpose reproduces hidden negative +gradients to below `2e-12` relative error and the exact-BP update to below +`2e-7` absolute error. Actual experiments never make this copy. + +## HFA-S1: bounded short screen + +Use the frozen Oral-A development split and the A2b setting: seed-0 ResNet-20, +the first 10,000 post-split training examples, 5,000 validation examples, +20 epochs, batch 128, ordinary CIFAR augmentation, cosine decay without +warmup, momentum 0.9, weight decay `1e-4`, and final validation evaluation +only. Output learning rate is 0.1. Cross hidden learning rate +`{0.01, 0.03, 0.1}` with feedback scale 1 and the single predeclared feedback +seed derived from model seed 0. A 32-example training-prefix alignment probe is +diagnostic only. + +Select maximum final validation accuracy, then lower estimated MACs, then +lower hidden learning rate. The short screen opens one full validation run only +if the selected trajectory is finite and reaches at least 50% validation +accuracy. For interpretation, the already observed matched short endpoints are +DFA `37.16%`, failed-v1 SDIL `41.98%`, and BP `74.94%`; these are comparisons, +not selectable thresholds. + +## HFA-S2: full validation baseline + +If HFA-S1 passes, train the selected HFA setting for 200 epochs on all 45,000 +development-training examples. Copy the A1 schedule exactly: batch 128, +momentum 0.9, weight decay `1e-4`, output base rate 0.1, and hidden base rate +selected by HFA-S1, with 10x drops at epochs 100 and 150 and no warmup. Evaluate +the same 5,000-example validation split only at the end. No recovery grid is +allowed. + +This is a baseline endpoint, not an advancement test for SDIL. A finite result +at or above 80% is considered a strong matched local baseline that any learned +hierarchical SDIL variant must compare against. A weaker result remains +reportable and cannot trigger additional HFA tuning. Neither outcome permits +test-set evaluation. + +## Accounting and stop rules + +- Record fixed feedback parameters, feedback-convolution MACs, local + correlation MACs, peak memory, wall time, and zero causal queries. +- All three HFA-S1 jobs must use the same clean git commit and must not touch + CIFAR-10 test data. +- No feedback scale, seed, optimizer, architecture, or extra learning rate is + added after the S1 endpoints are read. +- HFA-S2, if opened, copies the selected S1 hidden rate mechanically and does + not select on intermediate validation accuracy. + |
