diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-04-02 23:34:12 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-04-02 23:34:12 -0500 |
| commit | 2aaabd9a95386bf9f274cb9907ac6a5306171759 (patch) | |
| tree | fbc7f8e5f59ded66489b2141f6e1d41ded0dc43e /experiments/ep_baseline.py | |
| parent | 142bf87309ad8180770238ce097b0b1c9d33f5d7 (diff) | |
Fix EP credit sign: negate (h_nudge - h_free)/β to align with BP grad direction
EP nudge moves h toward lower loss (opposite to BP grad which points toward loss increase).
Without negation, Gamma is negative and rho is -0.25.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diffstat (limited to 'experiments/ep_baseline.py')
| -rw-r--r-- | experiments/ep_baseline.py | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/experiments/ep_baseline.py b/experiments/ep_baseline.py index de7d853..7f3d004 100644 --- a/experiments/ep_baseline.py +++ b/experiments/ep_baseline.py @@ -206,7 +206,8 @@ def ep_credit_signals(model, x, y, beta, T_nudge, alpha_nudge): _, h_free = model(x, return_hidden=True) h_nudged = ep_nudged_phase(model, x, y, h_free, beta, T_nudge, alpha_nudge) L = model.num_blocks - credits = [(h_nudged[l] - h_free[l]) / beta for l in range(L)] + # Negate: EP nudge moves h toward lower loss, opposite to BP grad direction + credits = [-(h_nudged[l] - h_free[l]) / beta for l in range(L)] return credits, h_free, h_nudged |