diff options
Diffstat (limited to 'docs/BASELINE_SPEC.md')
| -rw-r--r-- | docs/BASELINE_SPEC.md | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/docs/BASELINE_SPEC.md b/docs/BASELINE_SPEC.md index 5c5abc5..7c310b8 100644 --- a/docs/BASELINE_SPEC.md +++ b/docs/BASELINE_SPEC.md @@ -24,6 +24,13 @@ data/ckpts). 5000` (probes need the ladder). Results go into `docs/campaign/CASCADE_ABLATION_PLAN.md` as a RESULT entry (see existing entries for the format) + wandb. +## First task (warm-up, closes an open hole) + +The 72M BP twin is n=1 (best 3.2884, seed 1). Run seeds 2 and 3 with the exact original +config (`--L 12 --C 512 --H 8 --T 256 --B 24 --lr 1e-3 --opt muon --cosine --steps 234000 +--warmup 1000 --olmo2 --wd 0.1 --amp --save_every 5000`, seeds 2/3) — ~11-13h each on one +H100/A6000-class GPU. This sets the honest-gap denominator for every 72M claim. + ## Optional second baseline (sanity, not twin) A standard-recipe reference (AdamW β=(0.9,0.95), wd 0.1, cosine, GPT init) on the same |