diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:05:04 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:05:04 -0500 |
| commit | 2ec92906b45018665de97d6de63d6a754a1508d6 (patch) | |
| tree | d12b43b5ca1b4ee0ea94df55652be55ee86b483d /RESULTS.md | |
| parent | 45d2b57310d556c4d1e78ee185279eda14189585 (diff) | |
analysis: identify hierarchical spatial credit bottleneck
Diffstat (limited to 'RESULTS.md')
| -rw-r--r-- | RESULTS.md | 21 |
1 files changed, 21 insertions, 0 deletions
@@ -707,6 +707,27 @@ toward informative hierarchical/high-level feedback context or a mechanism that addresses remaining cross-layer/parameter-space noise; it does not justify another rate or warmup sweep. +The next oracle follows the actual residual DAG instead of adding global +context. On separate 128-example BatchNorm graphs, a local map from the exact +child error fields to each of the first six hidden populations gives: + +| held-out feedback family | early-third alignment | +|:--|--:| +| direct output, channel gated | 0.031123 | +| plus ordinary downstream activation context | 0.032069 | +| hierarchical 1x1 | 0.440525 | +| hierarchical 1x1 + child ReLU gate | 0.505210 | +| hierarchical 3x3 | 0.819917 | +| hierarchical 3x3 + child ReLU gate | 0.999767 | + +The nearly exact final row is an oracle reconstruction of the local transpose +convolution/ReLU computation, not a non-backprop learning result. Its value is +diagnostic: spatially aligned child error fields contain the missing early +credit, whereas ordinary downstream activations do not. A credible next method +must learn these hierarchical feedback maps without copying forward weights +and must compare against fixed/random hierarchical FA and BurstCCN; merely +adding recursive feedback is established prior art. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`. |
