diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:22:24 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 13:22:24 -0500 |
| commit | c24a6a575bce15e7d7da541ecb752c65b5e68e47 (patch) | |
| tree | 8166e8c7eebf2785cd811e4fbbc1578b185262bc /RESULTS.md | |
| parent | 225459f7ddfcd3c8d89f0515a0d766bf2a15c848 (diff) | |
results: close fixed hierarchical FA short gate
Diffstat (limited to 'RESULTS.md')
| -rw-r--r-- | RESULTS.md | 26 |
1 files changed, 26 insertions, 0 deletions
@@ -728,6 +728,32 @@ must learn these hierarchical feedback maps without copying forward weights and must compare against fixed/random hierarchical FA and BurstCCN; merely adding recursive feedback is established prior art. +### Matched convolutional hierarchical-FA baseline + +The HFA implementation was mechanically audited and its three-run short screen +was frozen before any HFA accuracy endpoint was observed. It mirrors the real +option-A residual DAG with independent random 3x3 feedback tensors and local +ReLU/BatchNorm Jacobians. An audit-only symmetric copy reproduces exact hidden +credit to `4.97e-16` relative error and the BP parameter update to `1.49e-8` +absolute error; actual runs never copy a forward tensor. + +On the identical 10k-example, 20-epoch ResNet-20 A2b setting: + +| hidden LR | validation accuracy | early-third alignment | estimated MACs | causal queries | +|--:|--:|--:|--:|--:| +| 0.01 | 39.64% | 0.004572 | 2.4242e13 | 0 | +| 0.03 | 41.58% | 0.011338 | 2.4242e13 | 0 | +| 0.10 | **43.52%** | **0.040406** | 2.4242e13 | 0 | + +All three trajectories are finite, use source commit `f37644e`, allocate +1.731 GB peak GPU memory, take 89.3--89.6 seconds on one GTX 1080, and never +evaluate test. Selected HFA improves on matched tuned DFA (`37.16%`) by 6.36 +points and failed-v1 SDIL (`41.98%`) by 1.54 points, but remains 31.42 points +below BP (`74.94%`). It misses the frozen 50% full-run threshold by 6.48 +points, so HFA-S2 is closed without further tuning. The result confirms that +spatial hierarchy is useful but that fixed random hierarchy alone does not +solve standard-depth credit. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`. |
