diff options
| author | Oscar Wan <oscarwan@oriong12.stanford.edu> | 2026-07-21 15:28:06 -0700 |
|---|---|---|
| committer | Oscar Wan <oscarwan@oriong12.stanford.edu> | 2026-07-21 15:28:06 -0700 |
| commit | ad4622e42abf2a55a37c5e522a6f8302e2e95cfd (patch) | |
| tree | ce668b916f9f16c4153c8b658bd772cd338ff90b /docs/campaign/FW135M_BP_BASELINE.md | |
| parent | 09c52fa97a946cf6dbdc500aef88da8522c2ce7d (diff) | |
Add portable 135M BP sweep handoff
Match the active EP configuration and support four-GPU BP candidates from collaborators' data locations.
Co-authored-by: Cursor <cursoragent@cursor.com>
Diffstat (limited to 'docs/campaign/FW135M_BP_BASELINE.md')
| -rw-r--r-- | docs/campaign/FW135M_BP_BASELINE.md | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/docs/campaign/FW135M_BP_BASELINE.md b/docs/campaign/FW135M_BP_BASELINE.md index 3f81ba8..2aade2c 100644 --- a/docs/campaign/FW135M_BP_BASELINE.md +++ b/docs/campaign/FW135M_BP_BASELINE.md @@ -12,8 +12,8 @@ Status: commands configured on branch `xiang`; no training has started. - Data: existing FineWeb-Edu 32k bins - Batch: `B24` - Target: `20N = 2,706,078,720` tokens -- Complete-batch exposure: `2,706,081,792` tokens -- Trainer argument: `--steps 440442` (440,443 inclusive updates) +- EP-matched complete-batch exposure: `2,703,366,144` tokens +- Trainer argument: `--steps 440000` (440,001 inclusive updates, matching `fw135m_bsign`) ## What changes from 72M @@ -29,6 +29,7 @@ Status: commands configured on branch `xiang`; no training has started. - FineWeb-Edu data and 32k tokenizer - OLMo2-style model implementation - Muon hybrid optimizer and cosine schedule +- 1,000-step warmup, matching `fw135m_bsign` - Weight decay 0.1 and BF16 autocast - Seed list and validation cadence - Existing `casc_bp_train.py` code path |