summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)Author
5 hoursAdd HRM-Orth v1 (codex round 2 Q6 pivot)YurenHao0426
Patch HRM Block with Lipschitz-bounded ops: - attention → cosine-normalized softmax attn - SwiGLU → OrthLinear (Cayley + weak diag scale) + MaxMin + OrthLinear - rms_norm + add → weighted residual (1-σ(w))·h + σ(w)·f(h) - Weak orthogonality: diag(s) with s_i ∈ [0.95, 1.0] for compression directions Keeps HRM ACT framework + H_level/L_level + cycles unchanged. Predicted +5-7pp vs SRM v1 (codex Q5 decomp): +1.5-2.5 (remove ReLU rank-kill via MaxMin) +2.0-3.0 (remove AOL attenuation via Cayley) +1.0-1.5 (orthogonal residual flow) Also adds: train_hrm_orth.py trainer, SRM v1 run logs, .gitignore ckpts/.codex Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
12 hoursAdd SRM training pipelineYurenHao0426
- config/arch/srm_v1.yaml: arch config for pretrain.py integration - scripts/train_srm.py: standalone from-scratch trainer based on step4 (HRM training infra adapted for SRM joint operator) The arch.yaml exposes κ, η, α, n_iters, n_aol_layers as Hydra params. train_srm.py adds joint Lyapunov diagnostic via JVP on srm_block to verify λ_1 ≤ log((1-α)+α·κ) per micro-step. Smoke tested with hidden=128, n_iters=4 on Sudoku 1k: empirical Lip 0.28 << bound 0.90. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>