From 7ca9658999c8f882b3d5d295a4201cc9a1ce9cde Mon Sep 17 00:00:00 2001 From: YurenHao0426 Date: Wed, 22 Jul 2026 13:10:14 -0500 Subject: baseline: add convolutional hierarchical feedback alignment --- THEORY.md | 1 + 1 file changed, 1 insertion(+) (limited to 'THEORY.md') diff --git a/THEORY.md b/THEORY.md index 676d28e..93ef88c 100644 --- a/THEORY.md +++ b/THEORY.md @@ -356,6 +356,7 @@ work `S_l=sum_{j>l} F_j`. |:--|:--|:--|:--|:--|:--| | BP | forward + reverse | exact transposed Jacobians | ordinary supervised loss | about one reverse pass | saved activations / graph | | FA | forward + reverse-like serial feedback | fixed random feedback | ordinary loss | `O(F)` feedback work | activations + feedback states | +| convolutional hierarchical FA | forward + residual-DAG feedback | independent random 3x3 feedback tensors; exact parameter-free shortcut graph | ordinary loss | one feedback convolution per non-stem forward convolution (`0.988x` forward MACs at ResNet-20, plus local correlations) | hidden feedback maps + forward caches | | DFA | forward + direct feedback | fixed random output maps | ordinary loss | `O(sum_l d_l d_out)` | activations + feedback states | | learned NP, simultaneous | ordinary forward; one duplicate clean plus `2K` noisy full forwards every `e` batches | learned direct maps; no weight transport | `2KB/e` per ordinary batch | `(1+2K)F/e` | up to `2KB` noisy examples in the vectorized implementation | | learned NP, layerwise | ordinary forward; clean baseline plus antithetic suffix replays | learned direct maps | `(1+2KH)B/e` | `(F + 2K sum_l S_l)/e`, quadratic in depth for balanced layers | serial suffix replay, no `2K` batch expansion | -- cgit v1.2.3