# FW135M BP Twin Run Sheet

Source of truth: `docs/BASELINE_SPEC.md`.

Status: commands configured on branch `xiang`; no training has started.

## First width-only rung

- Shape: `L12/C768/H12/T256`
- Head dimension: `64`
- Exact local parameter count: `135,303,936`
- Data: existing FineWeb-Edu 32k bins
- Batch: `B24`
- Target: `20N = 2,706,078,720` tokens
- EP-matched complete-batch exposure: `2,703,366,144` tokens
- Trainer argument: `--steps 440000` (440,001 inclusive updates, matching `fw135m_bsign`)

## What changes from 72M

- Width: `512 -> 768`
- Heads: `8 -> 12`
- Parameters: `72.11M -> 135.30M`
- Run length: increases to preserve approximately 20 tokens/parameter

## What stays identical

- L12 depth and 64-dimensional heads
- T256 and B24
- FineWeb-Edu data and 32k tokenizer
- OLMo2-style model implementation
- Muon hybrid optimizer and cosine schedule
- 1,000-step warmup, matching `fw135m_bsign`
- Weight decay 0.1 and BF16 autocast
- Seed list and validation cadence
- Existing `casc_bp_train.py` code path

## LR sweep

At this smallest new rung only:

- `7e-4`
- `1e-3`
- `1.4e-3`

Only Adam-side LR changes. Carry the winner to larger rungs.

No result may be quoted with fewer than two seeds. Record best validation CE, final-step CE, and tail median. Save every 5,000 steps and log validation every 100 steps.

## Dry-run launcher

From `ep_run/`:

```bash
python -m py_compile fw135m_baseline.py
python fw135m_baseline.py --mode smoke
python fw135m_baseline.py --mode sweep
python fw135m_baseline.py --mode full
```

Commands print by default. Add `--execute` only when ready.

Smoke:

```bash
python fw135m_baseline.py --mode smoke --execute
```

LR sweep:

```bash
python fw135m_baseline.py --mode sweep --execute
```

Selected full run:

```bash
python fw135m_baseline.py --mode full --lr <selected-lr> --seed 1 --execute
python fw135m_baseline.py --mode full --lr <selected-lr> --seed 2 --execute
```

W&B remains off until configured. Later:

```bash
python fw135m_baseline.py --mode full --lr <selected-lr> \
  --wandb_project <project> --execute
```

## Final EP/BP comparison

The EP arm must use the same shape, data, B/T, steps, optimizer family, selected shared settings, seeds, and evaluation cadence. Only the training rule and EP-specific `β` controls differ.