From 2a14f8ddc9821e6ce192021042d1c7c839bf0b00 Mon Sep 17 00:00:00 2001 From: Yuren Hao Date: Tue, 21 Jul 2026 11:50:39 -0500 Subject: =?UTF-8?q?BASELINE=5FSPEC:=20=E9=A6=96=E4=BB=BB=E5=8A=A1=3D72M=20?= =?UTF-8?q?BP=20seeds=202/3(=E8=AF=9A=E5=AE=9Egap=E5=88=86=E6=AF=8D,exact?= =?UTF-8?q?=20config=E7=BB=99=E5=87=BA);=20BP=E7=BA=BF=E5=85=A8=E6=9D=83?= =?UTF-8?q?=E7=A7=BB=E4=BA=A4=E5=8D=8F=E4=BD=9C=E8=80=85(=E6=9C=AC?= =?UTF-8?q?=E5=9C=B0bp=5Fs2=E5=B7=B2=E6=92=A4,GPU=E5=85=A8=E8=BD=ACEP)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_014FAPDWQ49M5Ye3NpTndTpn --- docs/BASELINE_SPEC.md | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'docs') diff --git a/docs/BASELINE_SPEC.md b/docs/BASELINE_SPEC.md index 5c5abc5..7c310b8 100644 --- a/docs/BASELINE_SPEC.md +++ b/docs/BASELINE_SPEC.md @@ -24,6 +24,13 @@ data/ckpts). 5000` (probes need the ladder). Results go into `docs/campaign/CASCADE_ABLATION_PLAN.md` as a RESULT entry (see existing entries for the format) + wandb. +## First task (warm-up, closes an open hole) + +The 72M BP twin is n=1 (best 3.2884, seed 1). Run seeds 2 and 3 with the exact original +config (`--L 12 --C 512 --H 8 --T 256 --B 24 --lr 1e-3 --opt muon --cosine --steps 234000 +--warmup 1000 --olmo2 --wd 0.1 --amp --save_every 5000`, seeds 2/3) — ~11-13h each on one +H100/A6000-class GPU. This sets the honest-gap denominator for every 72M claim. + ## Optional second baseline (sanity, not twin) A standard-recipe reference (AdamW β=(0.9,0.95), wd 0.1, cosine, GPT init) on the same -- cgit v1.2.3