diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-04-08 18:59:33 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-04-08 18:59:33 -0500 |
| commit | 6a057a379e58dc464f04e5208861699b01b5d477 (patch) | |
| tree | f91ca912d69c3b887aabc0cb93c22dc10eafb65a /protocol/protocol.py | |
| parent | 7c3b3393ca2264c463296144461d2ce6c62e3657 (diff) | |
paper v2.31.14: §3 ¶1 local-loss formula sign-consistency
Original §3 ¶1: "reducing a local loss of the form -<f_l, a_l>"
= minimizing -<f, a>
= maximizing <f, a>
= "alignment with credit target is rewarded" (matches prose)
But "reducing a local loss of the form -X" reads awkwardly because the
"-" sign in front of an inner product looks like a typo or extra
negative. Replaced with the equivalent positive form:
"maximizing a local objective of the form <f_l(h_l), a_l>"
This is mathematically equivalent (both give optimizer direction
∇W in the direction that increases <f, a>) but reads more cleanly.
The §3 ¶1 prose "any direction in which a larger block output
improves inner-product alignment with the method's fixed or learned
credit target is rewarded" now matches the formula directly:
maximizing <f, a> = rewarding alignment.
This relies on the convention e_T = one_hot - softmax (paper a_l is the
negative of the script's a_dfa). The script's positive-sign minimization
of local_loss = (f * a_dfa).sum() is the same operation under that
convention. If the convention is unfamiliar to a reader, the actual
direction (toward larger inner product with a fixed random direction)
is unambiguously stated by the prose.
Page layout preserved: 9 pages main, refs p10, 0 overfull boxes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'protocol/protocol.py')
0 files changed, 0 insertions, 0 deletions
