diff options
Diffstat (limited to 'docs/campaign/FW135M_BP_BASELINE.md')
| -rw-r--r-- | docs/campaign/FW135M_BP_BASELINE.md | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/docs/campaign/FW135M_BP_BASELINE.md b/docs/campaign/FW135M_BP_BASELINE.md index 3f81ba8..2aade2c 100644 --- a/docs/campaign/FW135M_BP_BASELINE.md +++ b/docs/campaign/FW135M_BP_BASELINE.md @@ -12,8 +12,8 @@ Status: commands configured on branch `xiang`; no training has started. - Data: existing FineWeb-Edu 32k bins - Batch: `B24` - Target: `20N = 2,706,078,720` tokens -- Complete-batch exposure: `2,706,081,792` tokens -- Trainer argument: `--steps 440442` (440,443 inclusive updates) +- EP-matched complete-batch exposure: `2,703,366,144` tokens +- Trainer argument: `--steps 440000` (440,001 inclusive updates, matching `fw135m_bsign`) ## What changes from 72M @@ -29,6 +29,7 @@ Status: commands configured on branch `xiang`; no training has started. - FineWeb-Edu data and 32k tokenizer - OLMo2-style model implementation - Muon hybrid optimizer and cosine schedule +- 1,000-step warmup, matching `fw135m_bsign` - Weight decay 0.1 and BF16 autocast - Seed list and validation cadence - Existing `casc_bp_train.py` code path |