diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-09 09:52:33 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-09 09:52:33 -0500 |
| commit | 32d1fdbfd0becef18351762d5d1d2924640a52be (patch) | |
| tree | 671841b24d3284cd4018f64903583d170dd289c2 /ep_run/lt_ep_train.py | |
| parent | 7e338ed314d7d73fdfaf32daa5bc105127a42c99 (diff) | |
cascade root cause found: tied readout + default N(0,1) embedding init = pathological top-CE stiffness once predictions sharpen (sigma_tok~76 vs GPT-standard 0.02 giving ~1.6); add --untie and --tok_init; diag arms A/B/C
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'ep_run/lt_ep_train.py')
0 files changed, 0 insertions, 0 deletions