summaryrefslogtreecommitdiff
path: root/ep_run/casc_eq_train.py
AgeCommit message (Collapse)Author
2026-07-19CODEX审计修复×4: ↵Yuren Hao
_legal门AND→OR+NaN非法+ddp_max(600步连续放行的逃生孔,主因); kretry救活持久化折半; up-step移到commit点后; bsimp进ckpt — codex裁决:主因=门代码缺陷,AIMD算术无错(0.01423≈ln1.01/ln2,392.37=1.01^600精确),用户'是bug'判定成立 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-19beta_simple(用户规格,复杂机构全拆): ↵Yuren Hao
β=起始×bsimp,干净步×1.01,非法步sync折半持久;σ缩放/floor/bf_late/cap/ride/servo/wsync全部不触β; 开火证书PASS(近边态锯齿0.8-2.4e-3,18刀,爬升段2.7×,零崩溃); fw72m_simple王冠已发射 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-19beta_servo: ↵Yuren Hao
贴上界伺服(亮区反演cap*=servo·cap_rho/ρ̂双向一步落点,暗区复用ride爬升;常数仅0.8安全系数) — 开火证书:plain s195000死亡区2.5k步CERT_PASS(β双向移动25/26行,best 3.5703→3.4260,零崩溃;贴边开销+50%墙钟为最恶劣工况); fw72m_servo王冠从头跑已发射(servo+ride4+cap_floor0+wsync3) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-18governor接线修复:res/rho非法判据常开(beta_cap_rho>0即生效),crow ↵Yuren Hao
n-3的18k垃圾步正是从'表只接beta_sync'这个洞进的账;用户判定=adaptive beta bug,成立 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-18wsync: 同步权重步接受(用户风格重设计,knee撤回) — ↵Yuren Hao
opt.step前快照params+动量,下步nudged relax过同一_legal门,非法即p←(p+snap)/2折半重放(预算复用beta_sync语义),折半到底整步回滚+skip;零新常数,模型函数零改动;42M引爆预检+死亡窗口重放已排c190后 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-18RESULT 49: ↵Yuren Hao
回路解剖=23段乘积近临界(下行×72/上行×13k/顶读1e-6),plain只热b8一段(h6 logit 94 vs cent 75)边际越线,灾变=硬底响应打伤b1(入口vjp 1.3→14.9); batch噪声对照0.39/0.45=Q1封口; --logit_knee 80实装+knee臂排c190后 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-18RESULT 43+44: crown-3 ↵Yuren Hao
死因=cap硬底钉在塌落天花板之上(变相wall-2); endgame A/B平局(3.4245/3.4244)估计器无罪; bpmix定位gap全在上半blocks(77%)attn/ffn均分,底半=0; --cap_floor补丁+fw72m_plain_nf无硬底190k重放在飞 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-18--bpmix: gap-source ablation (overwrite EP grads with true BP grads per ↵Yuren Hao
module group); 5-arm endgame ablation chained after A/B Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-17--beta_sync: synchronous acceptance (this-step relax telemetry gates the ↵Yuren Hao
commit; reject -> halve beta, retry same batch; failed trial doubles as ceiling measurement for the ride cap) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-17ride-v2 (b)/(c) flag-gated (--ride_ema/--ride_cool, default off); isolation ↵Yuren Hao
probe design: control|a|a+b|a+c|a+d|full at 72M mid-stage Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-17ride-v2: (a) floor-jump cap continuity (the 0.09 bug), (b) rho-EMA ↵Yuren Hao
decisions, (c) 200-step post-attack cooldown, (d) adaptive drift ceiling (--drift_adapt); smoke: continuous beta across floor jump Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-17save final ckpt at last step (fixed-eval protocol); fw72m_ride crown-3 ↵Yuren Hao
launched (plain+ride governor); ride/BP seed campaign on GPU0 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-16--beta_ride ceiling-riding governor (two-sided bcap: climb in calm, attack ↵Yuren Hao
near threshold); smoke: 3e-3 -> 2.4e-2 hover at ride cap, rho calm at 42M tail Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-16RESULT 29: qcomp8 tax-free (+0.0018) = T64 8-bit compute green light; ↵Yuren Hao
--centmirror ships (cos 1.000000000, 1.72x -> 1.39x) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-16Stage-0 gate #1b: --qcomp_bits (compute on DAC-grid weights, fp32 master = ↵Yuren Hao
T64 word-streaming / shadow accumulation); qcomp8/6/4 launched Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-15Stage-0 gate #1: --qup_bits (absolute-grid stochastic-rounding weight ↵Yuren Hao
quantization = finite analog cell levels); qup8/qup6/qup10 arms launched on GPU0 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-15centfast (doubled-batch +/-beta centered, cos 1.000000000 vs sequential, ↵Yuren Hao
DDP-verified) + est_late/est_late_at (tail-only centered); quiet bench: 1.72x -> 1.64x, est_late@80% -> ~1.13x amortized Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-14wandb reorganized: per-regime team projects (ept-tinystories-42m / ↵Yuren Hao
ept-fineweb-72m), auto project routing in trainers, split replay + reports Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-14wandb -> team entity eqprop-llm-training (trainers + full-history replay script)Yuren Hao
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-14bcap v2: absolute-scale gate on rho meter (fixes v1 noise-floor starvation), ↵Yuren Hao
gentler attack 0.85/floor 0.05; crown c2 relaunched from s215000 with cap armed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-14beta-cap-by-loop-gain controller (--beta_cap_rho): rho^ meter from sweep ↵Yuren Hao
residuals, cap OVERRIDES floor near the wall; smoke passed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-14Adaptive relax (--relax_tol): sweep-to-tolerance + rho-triggered geta ↵Yuren Hao
backtracking + final full-step graphed round (E-read identity fix: mixing leaked iteration residual into E, gn 1e5); kuse telemetry; paired smoke at wall ckpt: adaptive cos 0.771 vs legacy 0.754 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-13Estimator arms machinery: --est centered/richardson (two-pass, O(b^2) bias), ↵Yuren Hao
--muon_mom/--adam_b1 knobs; smoke passed (resume+centered cos 0.998) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-13Fix swallowed --wandb_run arg (replace-comment bug); wandb default-on intactYuren Hao
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-13wandb ON by default in both trainers (project ept-cascade); BP twin gains ↵Yuren Hao
--amp; fw72m_bp launched GPU1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-13fw72m crown run launched (NCCL DDP prod), exact resume (opt state), RESULT ↵Yuren Hao
17, shuffle+prep scripts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-13Manual DDP for cascade-EP: guard-synced (drift/kretry via global max, gn on ↵Yuren Hao
reduced grads), sigma broadcast, per-rank data RNG, None-preserving flat all-reduce, bitwise param sync-check, grad-equivalence test PASSED (cos 0.999999999 vs single-GPU big batch) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-13Stage-2 data pipeline: FineWeb-Edu sample-10BT -> 32k BPE -> bins; --data ↵Yuren Hao
flag on both trainers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-12bf16 mixed-precision (--amp): gate PASSED (cos 0.9682 vs fp32 0.9687), ↵Yuren Hao
trainer flag + 3-seed 4k A/B launched Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-12RESULT 13: bsign neutral at 4k (1.7382 vs 1.7314); 42M gap-chasing closed at ↵Yuren Hao
noise floor; pivot to Stage-2 pipeline + E-tier
2026-07-11RESULT 11: stage1b sealed (gap 0.050); beta ceiling unfound (3e-3 -> cos ↵Yuren Hao
0.9974); K exonerated on OLMo2; bf16 naive-cast dead at any beta (TF32 stays cost baseline); f3e3 CE-endpoint cont launched Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-10RESULT 10: epoch endpoints (BP 1.2509/1.2750 vs EP 1.4802, +0.22 erosion ↵Yuren Hao
cost); GENERATION GATE PASSED (casc_gen.py, coherent stories, no-backprop 42.75M); stage1b improved-recipe pair launched (Muon cosine via build_hybrid) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-10Sustained dose-response sealed (3e-4/5e-4/1e-3 -> 0.975/0.983/0.990); ship ↵Yuren Hao
--bf_late floor ramp; next-run recipe locked (OLMo2+Muon+late-ramp)
2026-07-10RESULT 6: wall-2 = EP-specific marginal under-convergence at the ↵Yuren Hao
contractivity edge (A skips16/B skips2/C skips1/D-BP clean); ship --kretry (retry marginal batches at K=kmax); launch OLMo2 parity matrix with parity-gated epoch autolaunch Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-10OLMo2-standard architecture (user directive): norm-after-sublayer RMSNorm ↵Yuren Hao
blocks, full-width QK-norm, RoPE(500k), SwiGLU, no-bias, untied head, final RMSNorm, 0.02 init, grouped wd, optional z-loss (plumbed through nudge+readout+gate for EP exactness); EP smoke 42.75M cos=1.0000 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-10AUDIT: retract confounded Muon verdict; tone down parity claims (n=3, ↵Yuren Hao
best-of-noisy-val); scope depth-tax claim to 4k horizon; guard-split skip telemetry (skd/skg); add BP control arm diag_D_bp (+--resume in BP trainer); record resume confounds Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-10Stage-1 epoch blowup @12100: sig story REFUTED (only +8%, cos fine till ↵Yuren Hao
after); leading indicator=drift-guard skips -> contractivity bifurcation in nudged relaxation (cascade Hopf wall); add resume/sig0/final_ln; A/B/C diagnostic launched
2026-07-10QK-norm: replace nn.MHA with explicit SDPA attn + --qk_norm (OLMo2-style, ↵Yuren Hao
analog-friendly); cancel epoch, insert 8-run qk validation, stage roadmap TinyStories-epoch->FineWeb-Edu->OLMo2
2026-07-10Add --cosine (warmup->cosine decay to 0.1x lr) to both cascade trainers for ↵Yuren Hao
full-epoch runs
2026-07-10K refuted as L12 lever (cos K-invariant K3==K8); BP 3-seed sealed 1.919; ↵Yuren Hao
mechanism = finite-beta SNR collapse; add --beta_floor/--beta_fixed + launch floor sweep Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09BP-free formal audit (5.6-sol): EP update verified local (108 bitwise ↵Yuren Hao
zero-influence checks); fixes — gate_every<=0 = true off switch, governor reaction now opt-in (--gate_govern, default observe-only), dFdtheta restored as self-sealing invariant-test surface; test_bp_free.py 4/4 green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09A0.4 precision gate (TF32 harmless cos 0.9946==fp32; pure-bf16 cos 0.9427) + ↵Yuren Hao
Muon hybrid optimizer (--opt muon) wired into both trainers; D1a flagship matrix launched (L12xC512, 3xBP + 3xEP + Muon arms) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train: exact mode (dtop_every=1) now DEFAULT — bisection showed ↵Yuren Hao
the v7 dedup cost ~4% CE at tuned lr; fast mode kept as opt-in. K3 verdict: exact equilibrium-EP 2.0481 vs BP 2.053+-0.004 = matched-tuning PARITY Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v7: two algorithmic dedups — d_top refreshed every other ↵Yuren Hao
round (O(beta^2) error), last-rebuild graphs reused by the theta-readout (kills one full graphed chain per step); fp32 untouched for fair BP twinning Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v6: graph-reuse fb (rebuild graphs feed next round's vjps), ↵Yuren Hao
--compile flag, tok-sigma amortized every 25 steps — 2.23 -> 2.68 it/s on 1080 (3.28x BP) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade root cause found: tied readout + default N(0,1) embedding init = ↵Yuren Hao
pathological top-CE stiffness once predictions sharpen (sigma_tok~76 vs GPT-standard 0.02 giving ~1.6); add --untie and --tok_init; diag arms A/B/C Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v4: quality-governed estimator — periodic measured ↵Yuren Hao
cos(EP,BP) drives K/beta (spend when quality drops, relax when abundant); guards reduced to sanity-only Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train v3: adaptive beta (tok-sigma stiffness compensation), ↵Yuren Hao
adaptive-K fb with contraction test, non-contraction bail, in-training cos(EP,BP) telemetry Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09cascade equilibrium solver solved: fb (forward-backward message passing) K=3 ↵Yuren Hao
beta=0.003 — gates 1.0000/0.9990/0.9946 across BP trajectory, L12 0.9999; trainer v2 single-sided EP readout + divergence guard (~5x BP) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
2026-07-09casc_eq_train: equilibrium-mode cascade-EP trainer (GS-reverse solver K ↵Yuren Hao
sweeps, two-phase ±β, EP readout at relaxed states) — the true-EP route C1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn
generated by cgit v1.2.3 (git 2.25.1) at 2026-08-18 17:46:17 +0000