diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 08:24:03 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-07-10 08:24:03 -0500 |
| commit | 4e0d730765a8287cdd574dc39b8dafc16006b266 (patch) | |
| tree | c3cb52316f904939a38c05d03426f193f8bcc799 /docs/paper/PAPER_A_OUTLINE.md | |
| parent | c81d6a5b80122dd7f53dc2ff876b296742bcc25b (diff) | |
OLMo2-standard architecture (user directive): norm-after-sublayer RMSNorm blocks, full-width QK-norm, RoPE(500k), SwiGLU, no-bias, untied head, final RMSNorm, 0.02 init, grouped wd, optional z-loss (plumbed through nudge+readout+gate for EP exactness); EP smoke 42.75M cos=1.0000
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
Diffstat (limited to 'docs/paper/PAPER_A_OUTLINE.md')
0 files changed, 0 insertions, 0 deletions