summaryrefslogtreecommitdiff
path: root/RESULTS.md
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 13:05:04 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-07-22 13:05:04 -0500
commit2ec92906b45018665de97d6de63d6a754a1508d6 (patch)
treed12b43b5ca1b4ee0ea94df55652be55ee86b483d /RESULTS.md
parent45d2b57310d556c4d1e78ee185279eda14189585 (diff)
analysis: identify hierarchical spatial credit bottleneck
Diffstat (limited to 'RESULTS.md')
-rw-r--r--RESULTS.md21
1 files changed, 21 insertions, 0 deletions
diff --git a/RESULTS.md b/RESULTS.md
index 9a6fa81..eb8feec 100644
--- a/RESULTS.md
+++ b/RESULTS.md
@@ -707,6 +707,27 @@ toward informative hierarchical/high-level feedback context or a mechanism
that addresses remaining cross-layer/parameter-space noise; it does not
justify another rate or warmup sweep.
+The next oracle follows the actual residual DAG instead of adding global
+context. On separate 128-example BatchNorm graphs, a local map from the exact
+child error fields to each of the first six hidden populations gives:
+
+| held-out feedback family | early-third alignment |
+|:--|--:|
+| direct output, channel gated | 0.031123 |
+| plus ordinary downstream activation context | 0.032069 |
+| hierarchical 1x1 | 0.440525 |
+| hierarchical 1x1 + child ReLU gate | 0.505210 |
+| hierarchical 3x3 | 0.819917 |
+| hierarchical 3x3 + child ReLU gate | 0.999767 |
+
+The nearly exact final row is an oracle reconstruction of the local transpose
+convolution/ReLU computation, not a non-backprop learning result. Its value is
+diagnostic: spatially aligned child error fields contain the missing early
+credit, whereas ordinary downstream activations do not. A credible next method
+must learn these hierarchical feedback maps without copying forward weights
+and must compare against fixed/random hierarchical FA and BurstCCN; merely
+adding recursive feedback is established prior art.
+
## How to run
`experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}`
Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`.