diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-08-03 17:49:49 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-08-03 17:49:49 -0500 |
| commit | 1d2d0338ae25de7d60756545dad692c2f833c532 (patch) | |
| tree | fcc4fcf6ea9855b0571283f9f6020242831a0e5c /docs/audit | |
| parent | 1e1d266db41f3acdcd647df71070982b2d1b057b (diff) | |
预写: 给Ben信中的"请教K"段(用户08-03决定跑完135M后征询)
策略: 把可能的软肋转为对定理作者的真诚请教。要点=给全数据(K=1余弦0.9999/配对差3e-5/seed散布
大两个数量级)+主动指出我方round与其round非同一单位故不宣称与其K≥4矛盾+明说不愿悄悄改用K=1
+把成本收益(2.7→1.6×)摆出但声明不以描述失准为代价。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs/audit')
| -rw-r--r-- | docs/audit/EP_READOUT_FLOOR_AUDIT.md | 21 |
1 files changed, 21 insertions, 0 deletions
diff --git a/docs/audit/EP_READOUT_FLOOR_AUDIT.md b/docs/audit/EP_READOUT_FLOOR_AUDIT.md index 7786b87..cc7a071 100644 --- a/docs/audit/EP_READOUT_FLOOR_AUDIT.md +++ b/docs/audit/EP_READOUT_FLOOR_AUDIT.md @@ -123,3 +123,24 @@ Hughes et al. Optica 2018、Kendall 2006.01981、2503.22810、2602.03670、Stoch > remedy there is to form it elementwise in higher precision rather than to substitute. > > A standalone checker is at [tools/contrast_floor_check.py], and it runs against any implementation. + +## 给 Ben 的信中"请教 K"段(预写, 用户 08-03 决定: 跑完 135M 后一并发出并征询他的意见) +> One question I would rather ask you than decide on my own. In our layered setup the free phase is a +> single forward pass, as it is for PCNs in your ImageNet paper, so the only thing that iterates is the +> nudged equilibrium. We transmit the nudge down the whole stack analytically in one round, rather than +> updating layers asynchronously, and with that transmission the number of rounds turns out not to +> matter: at a fixed checkpoint the cosine to the backprop gradient is 0.9999 at one round, and over a +> paired six thousand step comparison the final loss at one, two, and three rounds agrees to three parts +> in a hundred thousand, against a seed to seed spread two orders of magnitude larger. So in this +> architecture the equilibrium iteration appears to do no measurable work, and one round is the adjoint +> up to a correction of order beta that sits only in the parameter Jacobian. +> +> Your sensitivity result gives K at least four, with the optimum near the depth of the network, which +> makes sense for asynchronous updates where a round moves information about one layer. I do not think +> our round and your round are the same unit, so I am reluctant to read our result as contradicting +> yours, and equally reluctant to quietly report one round as though the iteration count were a free +> parameter. How would you report this? Is the round count a meaningful axis at all for an architecture +> whose free phase is exact by construction, or is the honest framing that the contribution sits in the +> locality of the update and the finite beta behavior rather than in the equilibrium computation itself? +> Dropping to one round would take our training cost from 2.7 times backprop to about 1.6, which is +> worth having, but not at the price of describing the method inaccurately. |
