From c24a6a575bce15e7d7da541ecb752c65b5e68e47 Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Wed, 22 Jul 2026 13:22:24 -0500 Subject: results: close fixed hierarchical FA short gate --- RESULTS.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) (limited to 'RESULTS.md') diff --git a/RESULTS.md b/RESULTS.md index eb8feec..d13a58c 100644 --- a/RESULTS.md +++ b/RESULTS.md @@ -728,6 +728,32 @@ must learn these hierarchical feedback maps without copying forward weights and must compare against fixed/random hierarchical FA and BurstCCN; merely adding recursive feedback is established prior art. +### Matched convolutional hierarchical-FA baseline + +The HFA implementation was mechanically audited and its three-run short screen +was frozen before any HFA accuracy endpoint was observed. It mirrors the real +option-A residual DAG with independent random 3x3 feedback tensors and local +ReLU/BatchNorm Jacobians. An audit-only symmetric copy reproduces exact hidden +credit to `4.97e-16` relative error and the BP parameter update to `1.49e-8` +absolute error; actual runs never copy a forward tensor. + +On the identical 10k-example, 20-epoch ResNet-20 A2b setting: + +| hidden LR | validation accuracy | early-third alignment | estimated MACs | causal queries | +|--:|--:|--:|--:|--:| +| 0.01 | 39.64% | 0.004572 | 2.4242e13 | 0 | +| 0.03 | 41.58% | 0.011338 | 2.4242e13 | 0 | +| 0.10 | **43.52%** | **0.040406** | 2.4242e13 | 0 | + +All three trajectories are finite, use source commit `f37644e`, allocate +1.731 GB peak GPU memory, take 89.3--89.6 seconds on one GTX 1080, and never +evaluate test. Selected HFA improves on matched tuned DFA (`37.16%`) by 6.36 +points and failed-v1 SDIL (`41.98%`) by 1.54 points, but remains 31.42 points +below BP (`74.94%`). It misses the frozen 50% full-run threshold by 6.48 +points, so HFA-S2 is closed without further tuning. The result confirms that +spatial hierarchy is useful but that fixed random hierarchy alone does not +solve standard-depth credit. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh "" "" `. -- cgit v1.2.3