diff options
| author | YurenHao0426 <Blackhao0426@gmail.com> | 2026-05-23 04:37:43 -0500 |
|---|---|---|
| committer | YurenHao0426 <Blackhao0426@gmail.com> | 2026-05-23 04:37:43 -0500 |
| commit | 27b69c2b75a251ec39ece318c26ccb1a25d9fd79 (patch) | |
| tree | 1d746fc806caca8b490ba282f4352e87a29fd411 /models/hrm | |
| parent | ac15626f8db096a63c775b84c9dc868776a6feda (diff) | |
Add SRM-Joint-AOL v1 model
Forked from HRM ACT framework. Replaces dual H_level/L_level transformer
stacks with a single joint operator T(h, l) that is provably contractive
under weighted P-norm with Lip_P(T) ≤ (1-α) + α·κ < 1.
Per-step Lyapunov bound: λ_1 ≤ log((1-α) + α·κ).
With κ=0.86 → λ_1 ≤ -0.15 ≈ HRM success regime (no CF needed).
Components:
- AOLLinear: 1-Lipschitz via Prach & Lampert rescaling (float32 normalization)
- AOLBlock: stack with ReLU (1-Lipschitz activation)
- CayleyOrthogonal: exact orthogonal cross-coupling
- BlockGain: softmax row-sum bound under P-norm
- AOLTokenMixer: 1-Lipschitz token + channel mixing
Smoke test passes: params=939k (hidden=256 test config),
forward OK, empirical Lip=0.17 < theoretical bound 0.90.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Diffstat (limited to 'models/hrm')
0 files changed, 0 insertions, 0 deletions
