summaryrefslogtreecommitdiff
path: root/.vscode
diff options
context:
space:
mode:
authorYurenHao0426 <Blackhao0426@gmail.com>2026-05-23 12:04:04 -0500
committerYurenHao0426 <Blackhao0426@gmail.com>2026-05-23 12:04:04 -0500
commitfe4d92760f9d9ce9d9f41eb0fe69dd9eadc1534c (patch)
tree33c95a3d7bcc5cbfc2233f60e3ed319aa4e07c9c /.vscode
parent152821462023690df5d2bf90812e1cb5b1ca7274 (diff)
Add HRM-Orth v1 (codex round 2 Q6 pivot)
Patch HRM Block with Lipschitz-bounded ops: - attention → cosine-normalized softmax attn - SwiGLU → OrthLinear (Cayley + weak diag scale) + MaxMin + OrthLinear - rms_norm + add → weighted residual (1-σ(w))·h + σ(w)·f(h) - Weak orthogonality: diag(s) with s_i ∈ [0.95, 1.0] for compression directions Keeps HRM ACT framework + H_level/L_level + cycles unchanged. Predicted +5-7pp vs SRM v1 (codex Q5 decomp): +1.5-2.5 (remove ReLU rank-kill via MaxMin) +2.0-3.0 (remove AOL attenuation via Cayley) +1.0-1.5 (orthogonal residual flow) Also adds: train_hrm_orth.py trainer, SRM v1 run logs, .gitignore ckpts/.codex Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Diffstat (limited to '.vscode')
0 files changed, 0 insertions, 0 deletions