diff options
| -rw-r--r-- | docs/audit/EP_READOUT_FLOOR_AUDIT.md | 21 |
1 files changed, 21 insertions, 0 deletions
diff --git a/docs/audit/EP_READOUT_FLOOR_AUDIT.md b/docs/audit/EP_READOUT_FLOOR_AUDIT.md index 7786b87..cc7a071 100644 --- a/docs/audit/EP_READOUT_FLOOR_AUDIT.md +++ b/docs/audit/EP_READOUT_FLOOR_AUDIT.md @@ -123,3 +123,24 @@ Hughes et al. Optica 2018、Kendall 2006.01981、2503.22810、2602.03670、Stoch > remedy there is to form it elementwise in higher precision rather than to substitute. > > A standalone checker is at [tools/contrast_floor_check.py], and it runs against any implementation. + +## 给 Ben 的信中"请教 K"段（预写, 用户 08-03 决定: 跑完 135M 后一并发出并征询他的意见） +> One question I would rather ask you than decide on my own. In our layered setup the free phase is a +> single forward pass, as it is for PCNs in your ImageNet paper, so the only thing that iterates is the +> nudged equilibrium. We transmit the nudge down the whole stack analytically in one round, rather than +> updating layers asynchronously, and with that transmission the number of rounds turns out not to +> matter: at a fixed checkpoint the cosine to the backprop gradient is 0.9999 at one round, and over a +> paired six thousand step comparison the final loss at one, two, and three rounds agrees to three parts +> in a hundred thousand, against a seed to seed spread two orders of magnitude larger. So in this +> architecture the equilibrium iteration appears to do no measurable work, and one round is the adjoint +> up to a correction of order beta that sits only in the parameter Jacobian. +> +> Your sensitivity result gives K at least four, with the optimum near the depth of the network, which +> makes sense for asynchronous updates where a round moves information about one layer. I do not think +> our round and your round are the same unit, so I am reluctant to read our result as contradicting +> yours, and equally reluctant to quietly report one round as though the iteration count were a free +> parameter. How would you report this? Is the round count a meaningful axis at all for an architecture +> whose free phase is exact by construction, or is the honest framing that the contribution sits in the +> locality of the update and the finite beta behavior rather than in the equilibrium computation itself? +> Dropping to one round would take our training cost from 2.7 times backprop to about 1.6, which is +> worth having, but not at the price of describing the method inaccurately. |