diff options
| author | YurenHao0426 <blackhao0426@gmail.com> | 2026-02-11 14:21:11 -0600 |
|---|---|---|
| committer | YurenHao0426 <blackhao0426@gmail.com> | 2026-02-11 14:21:11 -0600 |
| commit | c69a4c6e3596f75bd392c27d3c072adc825ce497 (patch) | |
| tree | 3d5c836a7adbf7e5f51e945c576e074fc13fd4cb /scripts/sanity_check.py | |
| parent | 039c12d3cf7178db6a7d80b02cf022d67231014e (diff) | |
- A12 (logit=3): NLL 2.76, A13 (logit=0): NLL 3.51, A14 (logit=1): NLL 3.26
- All worse than baseline (2.46). Lower init_logit = more deviation = worse NLL
- Confirms: gradient flows (gates move), but A=1 is global optimum for frozen model
- Added Dolma streaming retry logic (max 10 retries, 30s wait)
- Phase 1 frozen approach has fundamental limitation; Phase 2 (unfreeze) needed
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diffstat (limited to 'scripts/sanity_check.py')
0 files changed, 0 insertions, 0 deletions