summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
Diffstat (limited to 'docs')
-rw-r--r--docs/audit/EP_READOUT_FLOOR_AUDIT.md21
1 files changed, 21 insertions, 0 deletions
diff --git a/docs/audit/EP_READOUT_FLOOR_AUDIT.md b/docs/audit/EP_READOUT_FLOOR_AUDIT.md
index 7786b87..cc7a071 100644
--- a/docs/audit/EP_READOUT_FLOOR_AUDIT.md
+++ b/docs/audit/EP_READOUT_FLOOR_AUDIT.md
@@ -123,3 +123,24 @@ Hughes et al. Optica 2018、Kendall 2006.01981、2503.22810、2602.03670、Stoch
> remedy there is to form it elementwise in higher precision rather than to substitute.
>
> A standalone checker is at [tools/contrast_floor_check.py], and it runs against any implementation.
+
+## 给 Ben 的信中"请教 K"段(预写, 用户 08-03 决定: 跑完 135M 后一并发出并征询他的意见)
+> One question I would rather ask you than decide on my own. In our layered setup the free phase is a
+> single forward pass, as it is for PCNs in your ImageNet paper, so the only thing that iterates is the
+> nudged equilibrium. We transmit the nudge down the whole stack analytically in one round, rather than
+> updating layers asynchronously, and with that transmission the number of rounds turns out not to
+> matter: at a fixed checkpoint the cosine to the backprop gradient is 0.9999 at one round, and over a
+> paired six thousand step comparison the final loss at one, two, and three rounds agrees to three parts
+> in a hundred thousand, against a seed to seed spread two orders of magnitude larger. So in this
+> architecture the equilibrium iteration appears to do no measurable work, and one round is the adjoint
+> up to a correction of order beta that sits only in the parameter Jacobian.
+>
+> Your sensitivity result gives K at least four, with the optimum near the depth of the network, which
+> makes sense for asynchronous updates where a round moves information about one layer. I do not think
+> our round and your round are the same unit, so I am reluctant to read our result as contradicting
+> yours, and equally reluctant to quietly report one round as though the iteration count were a free
+> parameter. How would you report this? Is the round count a meaningful axis at all for an architecture
+> whose free phase is exact by construction, or is the honest framing that the contribution sits in the
+> locality of the update and the finite beta behavior rather than in the equilibrium computation itself?
+> Dropping to one round would take our training cost from 2.7 times backprop to about 1.6, which is
+> worth having, but not at the price of describing the method inaccurately.