diff options
| author | YurenHao0426 <blackhao0426@gmail.com> | 2026-02-09 14:40:31 -0600 |
|---|---|---|
| committer | YurenHao0426 <blackhao0426@gmail.com> | 2026-02-09 14:40:31 -0600 |
| commit | 80579d6cc254d337a23e71404ae7ecab1849d1e5 (patch) | |
| tree | bc6790229c20af516da662d7a4b7c8c7f1c4cb8c /src/training/schedulers.py | |
| parent | ef678d2e1ba70b1a9dadb78c73ed372f986aea13 (diff) | |
Layer 0 has no incoming edges structurally (no prior layers), but
receives the embedding as input. The cascading gate was killing its
outgoing edges (hard: g=0, soft: g=0.5), causing nll_hard to be
~2x worse than baseline. Fix: set g=1 for layer 0 nodes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diffstat (limited to 'src/training/schedulers.py')
0 files changed, 0 insertions, 0 deletions