diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 12:46:08 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-07-22 12:46:08 -0500 |
| commit | da109afa23146988188c3af910c6a6205f0c63d9 (patch) | |
| tree | 3bd096b1be9ad91a0be6ab0dc037568473cb0345 /RESULTS.md | |
| parent | ba50c8fc286f6071c688a12606216cfb871a7ee0 (diff) | |
analysis: localize early-layer feedback bottlenecks
Diffstat (limited to 'RESULTS.md')
| -rw-r--r-- | RESULTS.md | 20 |
1 files changed, 20 insertions, 0 deletions
@@ -663,6 +663,26 @@ per full hidden field, whereas structured targets are scored per channel-basis moment. No empirical 1133x variance-reduction claim is made from their raw ratio. +A no-training oracle audit then separates feedback capacity from estimator +efficiency on disjoint 32-example BatchNorm batches. Early-third alignment is +`0.05494` when each example/channel receives unconstrained optimal coefficients +for the existing `[1,tanh(h)]` spatial fields, but falls to `0.02397` when those +coefficients must be predicted by the actual output-error-linear A/G family. +The learned structured estimator reaches `0.00721`, about 30% of that +cross-validated family oracle. Thus the current basis is not incapable of +crossing the v2 threshold, but causal regression leaves a substantial gap and +the output-error-only coefficient map itself discards over half of the spatial +oracle's directional alignment. + +Adding fixed local-average and channel-mean somatic fields does not help +(`0.02246` early-third cross-validated alignment). An independent spatial +template reaches `0.04797`, but its prediction/target energy ratio is `0.458` +despite low cosine, a warning that the high-capacity map amplifies +out-of-sample error rather than solving credit assignment cleanly. These are +post-failure oracle diagnostics, not trainable-method results. They rule out a +naive basis expansion and point toward both better causal sample efficiency +and a more informative hierarchical/high-level feedback context. + ## How to run `experiments/run.py --mode {bp,fa,dfa,sdil} --dataset {mnist,fmnist,cifar10} --depth D --residual {0,1} --act {tanh,gelu,silu,relu}` Batteries: `experiments/run_v2.sh <ds> "<depths>" <res> <act> "<seeds>" <ep> <pfx>`. |
