diff options
Diffstat (limited to 'RESULTS.md')
| -rw-r--r-- | RESULTS.md | 26 |
1 files changed, 26 insertions, 0 deletions
@@ -728,6 +728,32 @@ must learn these hierarchical feedback maps without copying forward weights and must compare against fixed/random hierarchical FA and BurstCCN; merely adding recursive feedback is established prior art. +### Matched convolutional hierarchical-FA baseline + +The HFA implementation was mechanically audited and its three-run short screen +was frozen before any HFA accuracy endpoint was observed. It mirrors the real +option-A residual DAG with independent random 3x3 feedback tensors and local +ReLU/BatchNorm Jacobians. An audit-only symmetric copy reproduces exact hidden +credit to `4.97e-16` relative error and the BP parameter update to `1.49e-8` +absolute error; actual runs never copy a forward tensor. + +On the identical 10k-example, 20-epoch ResNet-20 A2b setting: + +| hidden LR | validation accuracy | early-third alignment | estimated MACs | causal queries | +|--:|--:|--:|--:|--:| +| 0.01 | 39.64% | 0.004572 | 2.4242e13 | 0 | +| 0.03 | 41.58% | 0.011338 | 2.4242e13 | 0 | +| 0.10 | **43.52%** | **0.040406** | 2.4242e13 | 0 | + +All three trajectories are finite, use source commit `f37644e`, allocate +1.731 GB peak GPU memory, take 89.3--89.6 seconds on one GTX 1080, and never +evaluate test. Selected HFA improves on matched tuned DFA (`37.16%`) by 6.36 +points and failed-v1 SDIL (`41.98%`) by 1.54 points, but remains 31.42 points +below BP (`74.94%`). It misses the frozen 50% full-run threshold by 6.48 +points, so HFA-S2 is closed without further tuning. The result confirms that +spatial hierarchy is useful but that fixed random hierarchy alone does not +solve standard-depth credit. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`. |