From ad4622e42abf2a55a37c5e522a6f8302e2e95cfd Mon Sep 17 00:00:00 2001 From: Oscar Wan Date: Tue, 21 Jul 2026 15:28:06 -0700 Subject: Add portable 135M BP sweep handoff Match the active EP configuration and support four-GPU BP candidates from collaborators' data locations. Co-authored-by: Cursor --- docs/campaign/FW135M_BP_BASELINE.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) (limited to 'docs/campaign/FW135M_BP_BASELINE.md') diff --git a/docs/campaign/FW135M_BP_BASELINE.md b/docs/campaign/FW135M_BP_BASELINE.md index 3f81ba8..2aade2c 100644 --- a/docs/campaign/FW135M_BP_BASELINE.md +++ b/docs/campaign/FW135M_BP_BASELINE.md @@ -12,8 +12,8 @@ Status: commands configured on branch `xiang`; no training has started. - Data: existing FineWeb-Edu 32k bins - Batch: `B24` - Target: `20N = 2,706,078,720` tokens -- Complete-batch exposure: `2,706,081,792` tokens -- Trainer argument: `--steps 440442` (440,443 inclusive updates) +- EP-matched complete-batch exposure: `2,703,366,144` tokens +- Trainer argument: `--steps 440000` (440,001 inclusive updates, matching `fw135m_bsign`) ## What changes from 72M @@ -29,6 +29,7 @@ Status: commands configured on branch `xiang`; no training has started. - FineWeb-Edu data and 32k tokenizer - OLMo2-style model implementation - Muon hybrid optimizer and cosine schedule +- 1,000-step warmup, matching `fw135m_bsign` - Weight decay 0.1 and BF16 autocast - Seed list and validation cadence - Existing `casc_bp_train.py` code path -- cgit v1.2.3