| Age | Commit message (Collapse) | Author |
|---|
|
- config/arch/srm_v1.yaml: arch config for pretrain.py integration
- scripts/train_srm.py: standalone from-scratch trainer based on step4
(HRM training infra adapted for SRM joint operator)
The arch.yaml exposes κ, η, α, n_iters, n_aol_layers as Hydra params.
train_srm.py adds joint Lyapunov diagnostic via JVP on srm_block to verify
λ_1 ≤ log((1-α)+α·κ) per micro-step. Smoke tested with hidden=128, n_iters=4
on Sudoku 1k: empirical Lip 0.28 << bound 0.90.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
|
- Soften exact Lipschitz claim — bf16 cast makes bound approximate (only exact in fp32)
- BlockGain init_diag 1.0 → 3.0: original gave 27% cross-coupling (mislabeled "minimal");
3.0 gives ~5% which is what "minimal" should mean
- Note torch.compile risk with linalg.solve in CayleyOrthogonal
Per codex review of v1.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
|
Forked from HRM ACT framework. Replaces dual H_level/L_level transformer
stacks with a single joint operator T(h, l) that is provably contractive
under weighted P-norm with Lip_P(T) ≤ (1-α) + α·κ < 1.
Per-step Lyapunov bound: λ_1 ≤ log((1-α) + α·κ).
With κ=0.86 → λ_1 ≤ -0.15 ≈ HRM success regime (no CF needed).
Components:
- AOLLinear: 1-Lipschitz via Prach & Lampert rescaling (float32 normalization)
- AOLBlock: stack with ReLU (1-Lipschitz activation)
- CayleyOrthogonal: exact orthogonal cross-coupling
- BlockGain: softmax row-sum bound under P-norm
- AOLTokenMixer: 1-Lipschitz token + channel mixing
Smoke test passes: params=939k (hidden=256 test config),
forward OK, empirical Lip=0.17 < theoretical bound 0.90.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
|
|
|
remove the incorrect comment
|
|
|
|
use bibtex syntax highlighting for citation in README.md
|
|
|
|
Fix typo in README.md
|
|
|
|
|
|
|
|
|
