summaryrefslogtreecommitdiff
path: root/scripts/slurm_a14.sh
AgeCommit message (Collapse)Author
2026-02-11A12-A14 init_logit ablation: confirm frozen OLMo cannot benefit from sparse ↵HEADmainYurenHao0426
topology - A12 (logit=3): NLL 2.76, A13 (logit=0): NLL 3.51, A14 (logit=1): NLL 3.26 - All worse than baseline (2.46). Lower init_logit = more deviation = worse NLL - Confirms: gradient flows (gates move), but A=1 is global optimum for frozen model - Added Dolma streaming retry logic (max 10 retries, 30s wait) - Phase 1 frozen approach has fundamental limitation; Phase 2 (unfreeze) needed Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>