| Age | Commit message (Collapse) | Author |
|---|
|
Co-authored-by: spokane-way <spokane@way>
|
|
The window_starts filter dropped windows shorter than stride,
silently skipping up to (stride-1) tokens at the end of the
validation set. Now includes all windows with >= 1 scoreable
token, and clamps the score start for short final windows.
|
|
|
|
* SOTA attempt
* Improve score on SXM
---------
Co-authored-by: spokane-way <spokane@way>
|
|
keep tok_emb.weight in fp16 during int8 export (kills the quant gap),
shrink MLP hidden to 992 to fit under 16MB, bump warmdown to 3600
and matrix LR to 0.06.
tested on 8xH100 SXM (2 seeds) and 8xH200 SXM (3 seeds).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
|
|
