diff options
| author | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:25:33 -0500 |
|---|---|---|
| committer | yurenh <blackhao0426@gmail.com> | 2026-08-31 18:25:33 -0500 |
| commit | 98d05c2f52527d31cf1872f8eda19d1b6d91bc5b (patch) | |
| tree | a7dbf3d6a8d30acf696e2a351972cc9fef568791 /README.md | |
| parent | d60d87e6cd3c1caf2187600b2c73035abf66dde6 (diff) | |
one-click resume-safe ladder runner (env-configurable)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GkgLsACEF6CCP7EUfA5fZe
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 11 |
1 files changed, 8 insertions, 3 deletions
@@ -20,10 +20,15 @@ pip install -e . # or: pip install -e .[dev] && pytest tests/ - `src/zbp_scaling/diagnostics.py` — per-block branch-gain rho_k profile and the NSR constant c(scale) - `configs/` — model sizes (60m/124m/350m/1b) x training arms (bp / zbp_n16 / zbp_n4) -## Run -Data prep runs **on the training node** (H200), not on a dev machine; `data/` and `runs/` are gitignored. +## Run (one click, on the H200 node) ``` -python scripts/prepare_data.py --tokens 3e9 --out data/fineweb # on the H200 node +./scripts/run_ladder.sh # all sizes x arms, sequential, resume-safe +SIZES="s60" ARMS="bp zbp_n16" TOKENS=3e8 ./scripts/run_ladder.sh # any subset / budget +DRY=1 ./scripts/run_ladder.sh # print the plan +``` +Data prep runs **on the training node** (auto-invoked if `data/` is empty); `data/` and `runs/` are gitignored. +``` +python scripts/prepare_data.py --tokens 3e9 --out data/fineweb # manual form torchrun --nproc_per_node=8 scripts/train.py --model configs/model/m124.yaml --train configs/train/zbp_n16.yaml ``` Global batch is fixed in the train config; per-rank micro-batch and accumulation adapt to world size. |