diff options
| author | Yuren Hao <yurenh2@illinois.edu> | 2026-08-05 06:40:24 -0500 |
|---|---|---|
| committer | Yuren Hao <yurenh2@illinois.edu> | 2026-08-05 06:40:24 -0500 |
| commit | d5158e64cad8a008794b0e61270c3b18821b41b7 (patch) | |
| tree | d0f63dcd1897eb8915dbf2fef062ed46c015d23b | |
| parent | 044973e2a78e0faa1296ac370cefbfc2f6feadd6 (diff) | |
EqOLion R4(真实轨迹判决): 冻结极因子几十步退相关至~0.53=慢漂移前提对符号模式不成立(极均衡
放大病态尾部,小σ奇异向量高速旋转); 子满秩探针NaN, 满秩单迭代仅平基线; 存活空间收窄到温启动
全NS(3-5×)或Occam臂(行/列AGC)升主候选; 限定=最早期轨迹为最难工况, 晚期重dump后再判。
附: --dump_mom探针实装(601步×3矩阵/276MB)。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
| -rw-r--r-- | docs/hardware/EQOLION_DESIGN.md | 11 | ||||
| -rw-r--r-- | ep_run/casc_eq_train.py | 17 |
2 files changed, 28 insertions, 0 deletions
diff --git a/docs/hardware/EQOLION_DESIGN.md b/docs/hardware/EQOLION_DESIGN.md index 5fc7127..7ba1577 100644 --- a/docs/hardware/EQOLION_DESIGN.md +++ b/docs/hardware/EQOLION_DESIGN.md @@ -67,3 +67,14 @@ sign(M) vs sign(polar(M)) 一致率**中位 0.700(0.647-0.816)**,谱偏斜 3. 已死:O(1) 秩一探针;同步锚。 **下一轮证伪**:真实动量的逐步轨迹(11M 短跑 dump 每步动量),在其上跑搭车/块探针追踪, 判据 = 追踪态的 sign 一致率显著超过 0.70 基线且开销入账。 + +**R4(2026-08-05,真实轨迹判决,601 步 C128 动量 dump)**: +- 基线复核:sign(M) vs sign(polar) 在轨迹上 0.67-0.73(与 ckpt 测量一致)。 +- **冻结极因子基线 0.52-0.53 ≈ 随机**——极因子的符号模式在几十步内退相关。"动量慢漂移⟹极因子慢漂移" + 的前提对**符号模式**不成立:极均衡把全部 σ 拉到 1,恰恰放大了病态尾部方向,而小 σ 方向的奇异向量 + 在噪声下高速旋转。**极因子本质上是高翻腾对象**(这同时解释了 Muon 为何有效与为何难追)。 +- 子满秩探针(k=d/8, d/4)步长未调导致 NaN(0.000 行);满秩单迭代勉强存活但只打平 sign(M) 基线。 +- **存活空间收窄**:(a) 温启动的完整 NS(每步全秩 2-5 次迭代 ≈ 从头的 3-5× 节省,非数量级); + (b) Occam 臂升为主候选——行/列 AGC 可能拿走谱均衡里"有用的部分"而不碰病态尾部; + (c) 限定:本轨迹取自训练最早期(步 100-600, LR 峰值区, 漂移最快的最难工况)——**晚期 ckpt 重 dump + 后再判死刑**。 diff --git a/ep_run/casc_eq_train.py b/ep_run/casc_eq_train.py index 7ded9d9..4a436d1 100644 --- a/ep_run/casc_eq_train.py +++ b/ep_run/casc_eq_train.py @@ -61,6 +61,9 @@ ap.add_argument('--gen_new', type=int, default=120) ap.add_argument('--probe_dgspec', type=int, default=0) # >0: M1 spectroscopy, value = n batches; exits before training ap.add_argument('--probe_gains', default='1,2,4,8,16,32,64,128,256') ap.add_argument('--probe_f64', action='store_true') # fp64 states+model in the probe: the fp-floor decisive arm # >1: per-STEP log-uniform dgain_top in +ap.add_argument('--dump_mom', type=int, default=0) # >0: record 3 Muon momentum matrices every step + # for N steps -> runs/momdump_{tag}.pt, then exit + # (EqOLion tracking-falsification input) ap.add_argument('--read_lin', action='store_true') # linear-form theta-read: cotangent = the stored d tensor (full # precision) instead of (z - o) (an fp32-ROUNDED copy of d); # algebraically identical via the read identity z - o = d @@ -1048,6 +1051,20 @@ for step in range(start_step, args.steps + 1): WSNAP['p'] = [p.detach().clone() for p in all_params] WSNAP['o'] = _clone_state(opt.state_dict()) opt.step(); sched.step(); opt.zero_grad(set_to_none=True) + if args.dump_mom > 0 and args.opt == 'muon': + # per-step momentum trajectory for the EqOLion tracking falsification: three + # representative block matrices (low / mid / top), CPU-cloned after each step. + om = opt.optimizers[0] + _mats = om.param_groups[0]['params'] + _sel = [_mats[1], _mats[len(_mats) // 2], _mats[-1]] + if 'MDUMP' not in globals(): + MDUMP = {'shapes': [tuple(p.shape) for p in _sel], 'traj': []} + MDUMP['traj'].append([om.state[p]['mom'].detach().float().cpu().clone() + for p in _sel if p in om.state and 'mom' in om.state[p]]) + if step >= args.dump_mom: + torch.save(MDUMP, f'runs/momdump_{args.tag}.pt') + print(f'[momdump] DONE {len(MDUMP["traj"])} steps -> runs/momdump_{args.tag}.pt', flush=True) + import sys; sys.exit(0) if args.qup_bits > 0: # STAGE-0 HW GATE: finite conductance levels. Snap every weight to an ABSOLUTE # per-tensor grid (range/2^bits) with stochastic rounding (unbiased) — emulates |
