| Age | Commit message (Collapse) | Author |
|---|
|
Muon矩阵组加解耦衰减--muon_wd, C512全程臂链在bp_s3后
预测冻结在启动前(见wd_arm.sh注释): P1 sig_late降>15%(基线521), P2 drift峰<0.015, P3 尾窗
3.4003±0.010内或更好。P1&P2中+P3不劣 => wd入270M+冻结配方; P3劣 => 降档或改Hyperball控σ。
背景: drift标度分析(08-07)显示末段均值近平坦(C^0.10)但峰值顶端加速(C768=0.041=C512的2.7×),
根因σ∝C增长, 外推C1536-2048触0.5守卫线; wd是已知洞(muon分支无衰减)与该风险的连接点。
附: --adam_b2实装+Adam电池5臂在农场(β2=0.95从未测过)。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
Optimizer子类适配器(合成损失closure桥接EP预填梯度), 13路全量回归冒烟PASS
修复过程记录: 首版适配器非Optimizer子类被LambdaLR拒; 重写时splice错序造成muon.py重复段
(旧类后定义胜出), 按结构图手术去重并断言全符号在位。三轴定位: GPU配方候选(追踪慢变统计=
R4幸存哲学+噪声鲁棒卖点), 硬件不合适(d²记忆), 偏置公理最差档(逐方向白化放大低方差偏置)。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
NS消融判决(摘NS代价0.45, nanoGPT版未调用NS之谜归他们), sign家族聚在5.13-5.25; k价值曲线在跑; 曲率通道死刑+偏置公理入档
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
逐坐标归一化把持久小偏置放大成满幅漂移, sign全家同罪(否决只挡
方差不挡偏置); 曲率通道=偏置磁铁(E[s·ĝ]=βc+δ/β, 污染被1/β放大333×), 存活=两档|β|锁相分离;
评分表增第三轴=偏置放大系数, 器件电池预注册排序重排预测(per-matrix优雅退化/sign满幅/Dither翻身)。
附: --dump_grad探针+cautlion已实装。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
实装+本地筛选电池发射
R4后设计原则=只追不翻腾的目标。SignLine核心洞察: 正行/列缩放在sign内是无操作, 作为脉冲幅度
在sign外才有效=因子化自适应的模拟原生形态; ConsensusLion=双漏率电容+符合门的噪声否决;
DitherLion=免费噪声做期望软sign。KFAC-lite(追平稳的活动协方差)列为下一轮。
9臂与stage A同协议本地筛选中; 发射纪律教训追加: 发射与等待必须分Bash调用(超时杀树会扫掉setsid)。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
OLion按论文算法框(慢动量+Nesterov混合→NS→逐元素sign→γ=0.2定标+解耦wd; betas取Lion默认
0.9/0.99并注明为假设); Adafactor2D=行/列因子化二阶矩+RMS-1裁剪(07-11审计的"模拟原生Adam");
build_alt与build_hybrid同分割(矩阵走被测规则+其余AdamW), sgdm例外全参数SGD(全局部基线)。
四路CPU冒烟全过(损失下降, gate cos 1.0000)。
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
— 用户等价LR假说命中并定位到单矩阵; Adam岛m/√v节流,对centered/fp32/K8全不敏感=EP头读共享结构; --head_lr_mult实装; 电池2(headmix诊断+hlr1.67/2.5补偿)在飞
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
--muon_mom/--adam_b1 knobs; smoke passed (resume+centered cos 0.998)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
17, shuffle+prep scripts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
cost); GENERATION GATE PASSED (casc_gen.py, coherent stories, no-backprop 42.75M); stage1b improved-recipe pair launched (Muon cosine via build_hybrid)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
|
Muon hybrid optimizer (--opt muon) wired into both trainers; D1a flagship matrix launched (L12xC512, 3xBP + 3xEP + Muon arms)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
|
